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ABSTRACT 

The material contained in this publication abstracts 
the major reports prepared by the Department of Research and 
Evaluation during the 1972-73 school year. The major purposes of the 
Research and Evaluation Department of the Dallas Independent School 
District are to provide useful information to decision-makers and to 
serve as an accountability agent. The process requires cooperative 
action by decision-makers (i.e.^ curriculum developers, teachers, and 
administrators) and evaluators. The evaluator has four broad 
obligations in the evaluation process: to focus on evaluative 
information to be provided; to collect, organize, and analyze this 
information; to administer evaluative activities; and to provide 
relevant evaluative feedback to decision-makers at all levels. To 
provide these evaluation processes, the District's Department of 
Research and Evaluation is organized into three branches. The first, 
System-Wide Testing, is responsible for the design and implementation 
of the District's system-wide norm- referenced and 
criterion- referenced testing programs.. The second, System-Wide 
Evaluation, performs the longitudinal and cross-sectional research 
and evaluation necessary to supply major District decision-makers 
with information about the overall functioning of the District's 
programs. The third. Developmental Project Evaluation, evaluates 
specific developmental projects to ascertain the effectsof those 
projects on developing specific student abilities in areas that 
generally need additional mediation to that which is provided by the 
District's general academic program. (Author/JM) 
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OVERVIEW OF THE DEPARTMENT OF 
RESEARCH AND EVALUATION 



The major purposes of the Research and Evaluation Department 
of the Dallas Independent School District are to provide useful 
information to decision-makers and to serve as an accountability 
agent. The process requires cooperative action by decision-makers 
(i.e., curriculum developers, teachers, and administrators) and 
evaluators. The decision-maker's role is to weigh the evidence 
provided hira by the evaluator and render judgment about what course 
of action to take in the situation confronting him. Because the 
decision-raaker generally does not have the time or technical skill 
necessary to gather and analyze the objective data necessary to 
make informed decisions, the evaluator provides him with the infor- 
mation concerning the reason action must be taken and alternative 
strategies that are open. Thus, it is essential that the evaluator 
know enough about the decision-making process and the information 
used by the educator in reaching a given decision to identify the 
scientifically sound and useful information needed to reach an ob- 
jective decision. Through the aforementioned process, the educator 
becomes accountable for his decisions. 

In implementing data collection and analysis activities, the 
evaluator shares an obligation with the educator; he must provide 
information which is valid and objective. Objectivity requires that 
the evaluator must be free to identify and investigate the view- 
points of a given decision-maker's clients, constituents, and other 
interested parties. The evaluator has four broad obligations in the 
evaluation process. He is obliged to focus on evaluative information 
to be provided; to collect, organize, and analyze this information; 
to administer evaluative activities; and to provide relevant evalua- 
tive feedback to decision-makers at all levels. 

To provide these evaluation processes, the District's Department 
of Research and Evaluation is organized into three branches. The 
first, System-Wide Testing, is responsible for the design and imple- 
mentation of the District's system-wide norm- referenced and criterion 
referenced testing programs. The second, System-Wide Evaluation, per 
forms the longitudinal and cross-sectional research and evaluation 
necessary to supply major District decision-makers with information 
about the overall functioning of the District's programs. The third. 
Developmental Project Evaluation, evaluates specific developmental 
projects to ascertain the effects of those proj^ects on developing 
specific student abilities in areas th^^f nB^ti^rally need additional 
mediation to that which is provided by^tjTie-J)istrict ' s general aca- 
demic program. 

The material contained in this publication abstracts the major 
reports prepared by the Department of Research and Evaluation during 



the 1972-73 school year. All of the listed reports have been stu- 
died and responded to by the Evaluation Committee of the Board of 
Education. If the reader is interested in obtaining a particular 
report in its entirety, he should refer to page 5 of this book for 
information concer ning ordering of reports. 
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William T. Denton, Ph.D. 
Director - 
Developmental Project Evaluation 



TITLE I TARGETED ACHIEVEJIENT 
IN READING PROGRAM 



Objectives of the Program : The primary objective of the Targeted 
Achievement in Reading Program is to aid educationally disadvantaged 
students to achieve at least one month's growth in reading for one 
month of instruction, using one of four reading systems. The four 
reading systems are: Hoffman Information System Reading Program 
(Hoffman), Behavioral Research Laboratories/Sullivan Reading Program 
(BRL/Sullivan) , South\/est Regional Laboratory Reading Program (SWRL) , 
and Listen, Look, and Learn Reading Program (LLL) • 



Purpose of the Evaluation ; The evaluation of the Targeted Achieve- 
ment in Reading Program was designed to determine which program or 
combination of programs best aids the educationally disadvantaged 
student to achieve at least one month's growth in reading for one 
month of instruction. 



Sample ; Thirty-eight Title I elementary schools were included in the 
study. The Listen, Look, Learn Reading Program was taught in the 
following five schools: Navarro, Carver, Tyler, Earhart, and Washington 
The Hoffman Information System Reading Program was taught in the follow 
ing seventeen schools: Austin, Harris, Colonial, Ray, Harllee, City 
Park, Frazier, Hassell, Thompson, Carr, Roberts, Brown, Rice, Lisbon, 
Wheatley, Lanier, and Pease. The Southwest Regional Laboratory Reading 
Program was taught, in the first two grades, in the following six 
schools; Dunbar, Allen, Bushman, Johnston, Crockett, and Kleberg. 
The Behavioral Research Laboratories/ Sullivan Reading Program was 
taught in the following ten schools: Travis, Douglass, Rhoads, Darrell, 
Bryan, Bowie, Young, Fannin, Jackson, and Arlington Park. Students 
from Budd and Arcadia Park were selected to be in the comparison group. 



Evaluation Design : 

Stratification Assignment (Schools) 0X^0 

0X^0 
OX3O 
OX4O 
OCO 

The four TARP reading systems were assigned to schools in accordance 
with Title I guidelines. X-^ represents the Hoffman reading program, 
X2 represents the BRL reading program, X3 represents the SWRL reading 
program, represents the LLL reading program, and C represents the 
comparison group. Variables observed at each point of observation (0) 
were the pretest and posttest standardized reading achievement tests. 
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The major evaluation questions of interest were; 



1. \^/hich of the four reading systems shows the largest vgain in 
reading achievement for first-grade students; for second- 
grade students; for third-grade students; and for fourth- 
grade students? 

2. Do students in the TARP Reading Program show gains in reading 
achievement comparable to those of the comparison group? 

3. Has the TARP project been successful in meeting its objective 
of one month's growth in reading for one month of instruction? 

Evaluation Results : 

1. The achievement results for the first grade show that students 
in the BRL reading program scored higher on the reading and 
language totals of the CAT than students in the other three 
reading programs. Slight differences were found between 
Hoffman, SWRL, and LLL students in the first grade. At the 
second-grade level, the results of the standardized achieve- 
ment tests show SWRL students made the largest achievement 
gains. The next-largest gains were made by BRL students, 
then Hoffman students, then LLL students. At the third-grade 
level, BRL students again showed slightly higher scores on 
both the reading and language totals. Hoffman students showed 
the next-largest gains. At the fourth-grade level, there 
appears to be little difference between the three reading pro- 
grams on reading total; however, on the language total, BRL 
students performed at a higher level than students in the 
other programs. 

2. In general, TARP students showed grade-equivalent gains for 
grades two, three, and four comparable to those of the non- 
Title I comparison schools on both the reading and language 
totals of standardized achievement tests. Due to the nature 
of the tests given in the first grade, it was not possible to 
calculate grade-equivalent gains. 

3. The average gain made by students in the TARP reading programs 
at the second-grade level was a full year's growth in reading 
achievement. The average grade-equivalent gain for third- 
grade students on the reading total of the CTBS was approxi- 
mately .9. The average grade-equivalent gain for students in 
the fourth-grade was approximately .8 for the reading total 
and approximately .9 for the language total. 
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Recommendations of the Evaluator: 



!• The successful results of the TARP project clearly point out 

the need for continuation of at least three of the four reading 
programs. The programs that this evaluation strongly recom- 
mends being continued are: the Hoffman Information Systems 
Reading Program, Behavioral Research Laboratories /Sullivan 
Reading Program, and Southwest Regional Laboratory Reading 
Program. 

2. The effect of continuation of only three of the four reading 
programs would necessitate implementation of at least one of 
these reading programs in five TARP school? where LLL is 
currently being taught. It is suggested that SWRL/BRL be 
implemented in three of these schools and that BRL be imple- 
mented in two of these schools. It is strongly recommended 
that these programs be randomly assigned to these five 
elementary schools. 

3. Due to the consistently successful nature of the Sl^L read- 
ing program, in grades one and two, it is recoimnended that 
the SWRL reading program be extended to the third grade in 
the six TARP schools in which SWRL is currently taught. 

4. The results of two years of evaluation of the TARP project 
show that it has helped in aiding students to make sub- 
stantial gains in reading achievement. Therefore, it is 
recommended that the Program be expanded to other elemen- 
tary schools where students exhibit reading deficiencies. 

5. During the 1973-74 school year, the TARP project should be 
expanded to include fifth-grade students in all the elemen- 
tary schools in which TARP is currently being implemented.. 

6. In an effort to ensure continued student success in reading, 
it is recommended that students remain in the TARP project 
until they meet the criteria of reading at or above grade 
level for two consecutive years. Currently, students are 
ineligible for additional reading instruction once they have 
shown that they are reading at grade level. This decision 
is currently made on the basis of only one test score. 
Students in the TARP project exhibit reading difficulties 
which will bear directly on their future learning styles. 

It is therefore imperative that we as educators be as 
certain as possible that the student does not require 
further supplemental reading instruction before declaring 
him/her ineligible for TARP instruction. 

Additional recommendations for the TARP project are included in the 
Educational Testing Service Auditor's Report. 
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Additional Information : 

72- 114 Evaluation Design for the Targeted Achievement in Reading 

Program, 1972-73, Robert J. Costello, 79 pp. 

73- 185 Evaluation of the Targeted Achievement in Reading Program: 

1972-73, Robert J. Costello, 56 pp. 
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TITLE I TARGETED ACHIEVEMENT IN READING PROGRAM: 
CRITERION-REFERENCED ANALYSIS 



Objectives of the Program ; The Dallas Independent School District's 
(DISD) criterion-referenced reading tests were administered to a sample 
of students in the Targeted Achievement in Reading Program (TARP) . 
These tests were designed to measure the extent to which students could 
perform tasks related to reading objectives specific to each grade 
level. The objectives from which these tests were designed were 
cooperatively developed by the Department of Research and Evaluation 
and District reading consultants and teachers. 

The rationale for such testing was to provide information about how well 
a sample of students in each of the four reading programs had mastered 
the appropriate grade-level objectives. 

Purpose of the Evaluation : The use of the DISD criterion-referenced 
tests was to provide additional indices of the relative success of the 
four reading programs which are included in the TARP program. 

Sample : Two classrooms from a stratified random sample of schools were 
selected for testing. The schools were stratified to include at least 
one school for each of the four reading programs included in the TARP 
program. For a complete listing of the sample, refer to the following 
table. 



School 



Classes 



Index of 
Deprivation 



Reading 
Program 



Johnston 
Fannin 



Earhart 
Roberts 
Pease 



4 
4 
4 
8 
4 



81.01 
67.84 
38.04 
42.02 
42.53 



LLL 



Hoffman 
Hoffman 
SWRL/BRL 
BRL 



Evaluation Design : 



First Grade 




Second Grade 



Same as First 
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Third Grade 



Fourth Grade 

X. represents students in the Hoffman reading program, represents 
students in the BRL/Sullivan reading program, represents students 
in the Listen, Look, Learn (LLL) reading program, and X^ represents 
students in the Southwest Regional Laboratory (SWRL) reading program. The 
variable defining the observation point (0) is the posttest criterion- 
referenced test. 

1. Do certain TARP reading programs show higher levels 
of performance than other TARP reading programs on 
criterion-referenced test measures? 

Evaluation Results : The four reading programs were not randomly 
assigned to schools according to degree of deprivation. The reader 
should interpret these results after reading the text. 

1. At grade one, on a total of 20 objectives, at least 

50% of the students tested answered at least 75% of the 
questions correctly on six objectives for the Hoffman 
program, ten objectives for BRL, nine objectives for 
SWRL, and seven objectives for LLL. 

2. At grade two, on a total of 18 objectives, at 

least 50% of the students tested answered at least 75% 
of the questions correctly on eight objectives for 
Hoffman, 14 objectives for BRL, nine objectives 
for SWRL, and five objectives for LLL. 

3. At grade three, on a total of 16 objectives, at 

least 50% of the students tested answered at least 75% 
of the questions correctly on two objectives for 
Hoffman, six objectives for BRL, and four objectives for 
LLL. 

4. At grade four, very few students mastered grade-four 
obj ectives. 

Recommendations of Evaluator : This report was not designed to include a 
comprehensive evaluation of the TARP project (see Research Report No. 
73-185) and, as such, no recommendations about particular reading pro- 
grams will be included. Hc'vzever, it is appropriate to include a recom- 
mendation about the future use of criterion-referenced test data in the 
TARP project. 



er|c 12 



x^o 

Same as Third 



1. Criterion-referenced tests should be administered both 
as a pretest and posttest of a 10% sample oi the TARP 
population for evaluation purposes during the 19 73-74 
school year. 



Additional Information ; 

71- 27 Evaluation Design for Targeted Achievement in Reading 

Program, 1971, Robert J. Costello, 61 pp. 

72- 68 Evaluation of the Targeted Achievement in Reading 

Program, 1972, Robert J. Costello, 115 pp. 

72- 114 Evaluation Design for Targeted Achievement in Reading 

Program: 1972-73, Robert J, Costello, 76 pp. 

73- 185 Evaluation of the Targeted Achievement in Reading 

Program: 1972-73, Robert J, Costello, 56 pp. 

73-186 Targeted Achievement in Reading Program: Criterion- 
Referenced Tests, 1972-73, Robert J. Costello, 36 pp. 
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TITLE I READING CLINICS 



O bjectives of the Program : The purpose of the Reading Clinics Program 
(RCP) was to offer clincial reading services for diagnosing and teaching 
Title I children with reading disabilities and to foster a more positive 
attitude within the children toward the reading process. 

Purpose of the Evaluation : The evaluation of the Reading Clinics Program 
was designed to determine if the Program was successful in producing 
reading gains not only within the Program, itself, but relative to an 
appropria(:e control group, and to determine the effectiveness of concomi- 
tant variables as predictors of pupil achievement for use as diagnostic 
and prescriptive purposes. 

Sample : Two hundred and sixteen pupils, on which total data were 
available, fromC. F. Carr, G. W. Carver, S. Lanier, J. Madison, and 
B. T. Washington Reading Clinics were used in the analyses. There were 
150 RCP fifth-grade, 15 fifth-grade waiting-list, and 59 RCP sixth- 
grade and seventh— grade pupils. 

Evaluation Design : The design used for the evaluation of the RCP 
consisted of a pretest-posttest design, schematically represented as: 

\ \ h °3 °4 

where the variables at each point of observation (0) consisted of the 
Gates-MacGinitie Reading Test (GMRT) and Comprehensive Tests of Basic 
Skills (CTBS) pretest (0^, 0^) and posttest (0^, 0^) subtests; 
represents the RCP treatment. The vocabulary and comprehension subtests 
of the GMRT and the vocabulary and language subtests of the CTBS served 
as the criterion variables in a regression analysis (linear) where the 
variables of pretest, age, sex, IQ, pupil ethnicity, teacher ethnicity, 
number-of-sessions-attended, A-Trait measure, and a deprivation index 
for the school attended served as independent predictor variables. The 
major evaluation questions of interest were: 

1. Were there achievement gains within the grade levels 
served by the RCP on objective tests such as the GMRT 
and CTBS; were there achievement gains relative to an 
appropriate control? 

2. Were the concomitant variables significant predictors 
in pupil achievement , account ing for variances useful 
for diagnostic and prescriptive purposes? 
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Evalua t^i -^ n Results ; The results were discussed by test. 



1. The analysis of the GMRT data for 1972-73 indicated general 
ineffectiveness of the RCP when longitudinal performance 
considerations of the GMRT data from 1969-1973 were viewed. 
The 1972-73 evaluation indicated chat, although significant 
gains were made within the fifth, sixth, and seventh 
grades, the magnitude of the gains was not larger than would 
have been expected without treatment. Moreover, when 
achievement was considered relative to an appropriate 
control, there was no difference on either the vocabulary 

or comprehension subtests. This finding occurred when 
the fifth-grade RCP pupils were compared to pupils who 
remained on the fifth-grade RCP waiting list. Project 
administrative error permitted pupils on the RCP waiting 
list for sixth and seventh grades to be admitted to the 
RCP.. Consequently, appropriate control groups were not 
available for comparison. However, a longitudinal con- 
sideration of data (1969-72) indicated that, across 
grades four, five, and six and across both the vocabulary 
and comprehension subtests of the GMRT, there was a 
decrease in grade-equivalent gain from year to year. 

2. The analysis of the CTBS data indicated consistent 
ineffectiveness of the RCP for the 1972~73 data as well 
as the longitudinal (1971-72) data. Specifically, the 
results indicated that there were no significant gains on 
any subtests within any of the grade levels. However, 
there was a significant achievement decline for the 
vocabulary subtest within fifth grade. Moreover, for the 
fifth grade, there was no significant difference between 
RCP participants and children on the RCP fifth-grade 
waiting list, indicating a failure of the RCP Program. 
The longitudinal data (1971-72) also indicated that there 
previously were no achievement gains on the CTBS for RCP 
participants. 

3. The 1971-72 evaluation indicated that the concomitant 
variables of sex, ethnicity, IQ, and self-image were 
significant predictors of pupil achievement. The current 
evaluation demonstrated that student ethnicity, teacher 
ethnicity, KJ, number-of -sessions attended, pretest, 
deprivation index, and basic vocabulary index were 
significant predictors. 

Recommendations of Evaluator ; 

1. Based upon longitudinal (1971-72) and current data 

(1972-73), it is recommended that the Title I Reading 
Clinics Program, in its current form, be discontinued. 
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2. An alternative to discontinuance of the Title I RCP would 
be to orient the Program directly toward the needs of the 
disadvantaged reader with individually designed treatments. 
A possible scheme for this approach would consist of in- 
puts from a team of DISD professionals. The team would 
consist of the reading specialist, a clinical psychologist, 
and an educational psychologist evaluator. The function of 
the clinical psychologist would be diagnostic and treatment 
prescription; the function of the educational psychologist 
(evaluator) would be treatment prescription and evaluation; 
the function of the reading specialist would be reading 
diagnosis and treatment administration. 

3. The basic vocabulary index concomitant variable demonstrated 
diagnostic capability. In addition, since it accounted for 
a large part of the variance for pupil comprehension, in- 
creasing the pupils' vocabulary may be a good starting place 
in treatment. 



Additional Information ; 

72- 107 Evaluation Design for the Title I Reading Clinics, 1972-73, 

James P. Papay, 22 pp. 

73- 147 Evaluation Report: Title I Reading Clinics, 1972-73, James 

P. Papay, 64 pp. 
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ESEA TITLE I MULTIAGE PROGRAM 



OMectives of the ProRram : The Multiage Classes Program (MAP) had as 
its primary goal the education of five-, six-, and seven-year-old children 
(kindergarten through two) through a philosophy of nongraded, Multiage classes 
and an individualized, custom-tailored curriculum based upon performance 
objectives, self-pacing, and work-agreement schedules. 

The global goals of the MAP included the desire to: (a) help the child 
develop a more positive self-concept by offering educational experiences 
to him within a framework of success; (b) help each child develop the 
skill of making choices which will, in turn, lead him to accept responsi- 
bility for his own learning; (c) subject the child to less academic 
learning experiences; (d) help each child develop pride in the successful 
completion of his learning tasks; (e) help the child move naturally from 
the concrete to the abstract task through a designed curriculum; (f) 
demonstrate that the success of the Program resides with the teacher. 

Purpose of the Evaluation : The evaluation of the MAP was designed to 
determine the effectiveness of the MAP with a Title I population compared 
to a traditional program that included the Targeted Achievement in 
Reading Program (TARP) ; to determine the effectiveness of selected MP 
curriculum; to determine the effects of anxiety on pupil achievement; and 
to scrutinize the effects of teacher parameters, such as ethnicity, 
experience, and certification, on pupil performance. It should be remem- 
bered that the control groups received TARP reading instruction and tra- 
ditional instruction in other academic areas. 



Sample : The criterion-referenced component of the 1972-73 evaluation 
employed 557 pupils, on which complete data were available, from four 
schools selected from intervals along the full spectrum of deprivation 
indices for Title I schools. The schools used in the analyses consisted 
of G. W. Carver, P. Tyler, J. Rhoads, and J. Bryan. All testing for the 
criterion-referenced evaluation was performed independently of Research 
and Evaluation (except for- supervision) by ten substitute teachers trained 
specifically for the task by Research and Evaluation personnel. The teachers 
were selected from the approved DISD substitute teachers' list, generally 
on the basis of previous testing experience, early childhood experience, 
or clinic-type experience. 

For the evaluation of the objective data using the Boehm Test of Basic 
Concepts (BTBC) , the California Achievement Tests (CAT), and the Metro- 
politan Readiness Tests (MRT) , approximately a 34% random sample of ^lAP 
pupils was used for analyses purposes. The only stipulations placed on 
the random sample was that pupils had to have complete data and not have 
also participated in the Bilingual-Multicultural Education Program (BMEP) . 
Two hundred and six TARP program pupils were drawn from among the follow- 
ing schools: G. W. Carver, P. Tyler, N. W. Harllee, C. F. Carr. 0. M. 
RO'berts, C. Rice, J. Rhoads, J. H. Brown, J. N. Bryan, and E. M. Pease. 
Consequently, the MAP and TARP program children had similar deprivation 
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indices. It should be noted that the TARP program consisted of various 
reading programs. 



Evaluation Design : The evaluation design used for the criterion-referenced 
evaluation consisted of the following counterbalanced paradigm: 



Ol ^2 ^1 ^2 ^3 °3 ^4 ^5 '^6 °4 
Ol °2 ^4 ^5 ^6 °3 ^1 ^2 ^3 °4 



The observations (0 , 0^) consisted of the pretask A-Trait and A-State 
(anxiety) measure; the tests (T^^ , T^, T^) consisted of the concrf^te, semi- 
concrete, and abstract subtests for the communications module; 0 consisted 
of the Task 1 A-State measure; tests (T^, T , Tg) consisted of the concrete, 
semi-concrete, and abstract subtests from the mathematics module; and 0^ 
consisted of the Task 2 A-State measure. Counterbalancing the order for 
tests was undertaken to eliminate any practice effects across modules. 

Various repeated measures analysis of variance designs were used to assess 
the effects of anxiety on pupil achievement. These designs v/ere configured 
dependent upon the particular hypotheses tested. 

The design used for the evaluation of the objective data consisted of a 
pretest-posttest design whi.ch may be schematically represented as: 



0^ 0, 



h h ^2 



The observations (O-j^, consisted of the pretest and posttest data, 

respectively; consisted of the MAP treatment ^and X2 consisted of the 
TAUP program treatment. 

The regression analysis approach was employed for the BTBC component of 
this analysis. Within the regression analysis framework, the posttest 
BTBC served as the criterion variable, and, in order to reduce error 
variance, age and the pretest BTBC served as covariates. Predictor 
variables such as program type, pupil ethnicity, teacher ethnicity, 
teacher experience, teacher certification, and attendance were used to 
categorize the data to test hypotheses on their effectiveness in account- 
ing for pupil achievement. The major evaluation questions of interest 
were: 

1. Were the sampled curriculum modules from the MAP curriculum 
effective, and were MAP children superior to TARP program 
children on these materials as expected? 

2. Were low A-Trait and A-State pupils able to demonstrate greater 
performance than high A-Trait and A-State pupils on the criterion- 
referenced materials as predicted by State-Trait Anxiety theory? 
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3. 



Were there achievement differences between MAP and TARP children 
on the objective BTBC? 



4. Were teacher variables such as ethnicity, experience, and certi- 
fication important factors for pupil achievement within MAP? 

5. Were there differences between children taught by the MAP and 
TARP prograras on the MRT and CRT objective tests? 

6. Was att4sndance a significant factor in pupil achievement? 

-« 

7. Were kindergarten through one Multiage classes equivalent to single- 
ag^-g^ade kindergarten and one classes for learning, and were there sex 
differences within these classifications? 

8. Was the Program Implemented as specified? 



Evaluation Results : 

I. The criterion-referenced evaluation indicated that TARP 
progran children performed higher than MAP children on 
four of the six subtests on MAP curriculum; the differ- 
ence was significant only for the abstract concepts* 
subtest of the communications module* Children in the 
TARP program were not previously exposed to the criterion- 
referenced materxals while MAP children saw the materials 
in cneir classroonis as part of the MAP activities, were 
tested with the materials, brought to criterion, and retest- 
ed. Typically, the use of criterion-referenced materials 
has been shown to reduce pupil variance* The current 
evaluation found that TARP children obtained lower test 
variances than ^!AP children using the criterion-referenced, 
multiage curriculum. 

2* The results of the analysis of the standardized tests. 
Metropolitan Readiness Tests (MRT), and the California 
Achievement Tests (CAT) data indicated either higher scores for 
TARP pupils or findings of no difference when the MAP was com- 
pared with the TARP program. 

3* The results Indicated that the TARP kindergarten program children 
scored higher than the Multiage kindergarten children on the 
MKT although the Multiage kindergarten children showed initially 
superior pretest MRT scores (TARP kindergarten children were 
those in a traditional kindergarten who would probably enter a 
TARP clasji in the first grade)* 

4». When student ethnicity and teacher ethnicity were considered 
across programs, the standardized test data from the BTBC 
indicated that the TARP program children outperformed MAP 
children on the following classifications: 
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1) An^lo students taught by Anglo teachers, 

2) Negro students taught by Anglo teachers^ 

3) Mexican-An\erican students taught by Anglo teachers, and 

4) Mexican-American taught by Negro teachers. 

5. Student achievement was not necessarily facilitated by 
matching the student's ethnicity to the teacher. For 
Anglo and Mexican-American students, teacher ethnicity ' 
had no significant difference. 

6. Teacher certification was demonstrated not to be a 
significant factor for pupil achievement. 

7. Teacher experience within MAP (po years vs. one + years) 
was shown to affect pupil achievement significantly. 

8. Useable data were available only on two categories of 
aides and class sizes: one aide for class size less than 
or equal to 30; or, one aide for class size less than 

or equal to 60 but greater than 30. Both these classi- 
fication categories of aides and class sizes resulted in 
significant pupil achievement. 

9. Evaluation of adjunct data from the Bilingual Multicultural 
Education Program (BMEP) by Murray (see Report No. 73-142) 
indicated that there were no differences on criterion- 
referenced BMEP posttest data for Bilingual program children 
compared with children from the Multiage BMEP classes. 
However, Murray indicated that, although the MAP children 
started out on the pretest with superior scores, the 
superior advantage vanished over the year when compared 
with self-contained Bilingual program. This result corro- 
borated a previously reported similar finding which was 
observed when the MAP kindergarten children were compared with 
the TARP kindergarten children on the MRT standardized test. 

10. Murray (1973) also reported no sex differences for his data 
when comparing single-age and Multiage classes. 

11 . Process evaluat ion data indicat ed that the MAP was rated as 
individualized by the teachers. The MAP appeared to be 
implemented along the guidelines specified. 

12. The evaluation demonstrated that low A-Trait and low A-State 
(anxiety) children consistently out-performed the high A- 
State and A-Trait children. The educational implications 

of this finding revolved around (1) grouping strategies of 
Multiage children for optimal educational effect, (2) 
allocation of resource strategies (e.g., tea'^her aides), 
(3) selection of individual learning stratt ^es, and (4) 
curriculum development strategies. 



Recommendations of Evaluator : Before considering recommendations, 

it was also important to consider the overall results from the 1971-72 
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evaluation. The objective data from the Metropolitan Readiness Tests, the 
California Achievement Test-s, the Boehm Test of Basic Concepts, and a Self- 
image Test all indicated, overall, either smaller gains when the MAP was 
compared with a TARP program or no differences. Recommendations were made 
from these data not to extend the Program until curriculum revision was 
undertaken on a systematic basis. The results of the current evaluation 
suggested that systematic revision, and especially the assessment of the 
entry levels of the MAP materials, did not produce the intended effects. 
In a program such as MAP, which relies so heavily upon curriculum, the 
potential merit of an individualized, self-paced, multiage^grouping philos- 
ophy had to be evaluated largely in terms of curriculum effectiveness, 
especially in terms of producing transferable skills and knowledges. The 
implications of this situation were that a curriculum was evaluated, not 
a process, since the curriculum and the process could not be separated. 

The following recommendations were made toward improvement of the MAP: 

1. The Multiage curriculum strongly appears to be in need 
of systematic revision. 

2. Given the DISD desire for extending the Multiage classes 
concept, it is suggested that a variety of curriculum 
treatments be employed within the Multiage classes frame- 
work. Such curricula might include expanded use of the 
Houghton-Mifflin or Economy Basal Reading Series, success- 
ful TARP curricula, or other available systems focusing 
not only on reading but on other academic areas. 

3. Since teacher variables such as teacher certification, 
ethnicity, and experience had little programmatic impact 
on pupil achievement, added emphasis again was suggested 
toward multiage grouping and individualized instruction 
should be implemented. 

4. Within the context of multiage grouping, perhaps grouping 
by ability and/or on anxiety level in addition to age would 
have meaningful implications and should, perhaps, be tried 
out in selected classrooms. 

5. Additional aides would be useful, especially in teaching 
classrooms with predominantly high-anxious children, as 
suggested by the results. Moreover, other possibilities 
such as a more structured learning environment, provision 
of memory support materials, more feedback, and more 
individual attention may be warranted for instruction of 
high-anxious children. 

6. Since the comparison schools consisted of TARP schools, 
this report provides additional support for the effec- 
tiveness of the TARP reading progrsms. In a Title I 
setting, the Hoffman Information Systems Reading Program, 
Behavioral Research Laboratories/Sullivan Reading Program, 
and the Southwest Regional Laboratory Reading Program 
should be tried in a multiage environment. 
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7. Since this evaluation attempted to measure the MAP and 
TARP curricula only with economically and educationally 
deprived pupils, it is recomraended that the 1973-74 
evaluation determine the effects of MAP, TARP, and other 
appropriate curriculum materials with populations other 
, than the economically and educationally deprived. 

Additional Information : 

73-158 Multiage Classes Program, 1973, James P. Papay, 68 pp. 

72-98 Evaluation Design for the Multiage Classes Program: 

1972-73, James P. Papay, 24 pp. 
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BILINGUAL-MULTICULTURAL EDUCATION 



Objectives of the Program ; The Bilingual-Multicultural Education 
Program (BMEP) consists of four components: Instructional, Materials 
Acquisition, Staff Development, and Community Participation. 

The major objectives of the Instructional component are to allow the 
children to acquire basic concepts in the language they understand best, 
to instill in the children a sense of personal pride and integrity with 
respect to their cultural heritage, and to provide all children in the 
Program with an opportunity to become bilingual. The major function of 
the Materials Acquisition component is to support other components by 
procuring, adapting, and developing materials for the Bilingual class- 
room. The objective of the Staff Development component is to develop a 
cadre of highly trained personnel in Bilingual Education. The objectives 
of the Community Participation component are to enhance parental 
participation in Program development, teaching and learning, and Program 
monitoring. The Community Participation component also strives to in- 
crease, community understanding of the goals and accomplishments of the 
Program. 

Purpose of the Evaluation : The evaluation of the Bilingual-Multi- 
cultural Education Program was designed to determine the extent to which 
the Program objectives were met and to provide Program administrators 
with informatioii which could be used for the improvement of the 
effectiveness of the Program. 

Sample : Thirteen elementary schools and one high school were included 
in the study. The Bilingual-Multicultural Education Program was imple- 
mented at Allen, Douglass, Houston, Juarez, Lanier, Maple Lawn, Milam, 
Travis, St. Anne, St. Mary of Carmel, and Holy Trinity Elementary Schools 
and at North Dallas High School. Allen, Bonham, and Knight Elementary 
Schools served as comparison schools for the evaluation. Throughout the 
year, data were collected from BMEP children, parents, and teachers. 



Evaluation Design : The figures below schematically represented the 
evaluation designs used in the BMEP. Randomization of students or teachers 
to the BMEP was not used. Teachers were selected by language ability 
and/or a strong commitment to the philosophy of bilingual education. In 
general, ail students attending a school selected for bilingual education 
were included in the Program. In four of the eight public BMEP elementary 
schools, several classrooms were not included in the Program to 
accommodate children whose parents did not desire this type of education 
for their child or for children transferring from nonbilingual schools. 

1. 0^ X O2 
°1 °2 
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This design was used in the evaluation of the Instruction- 
al- and^'Staff Development component. X represented the 
implementation of the BMEP while 0^ and O2 represented, 
respectively, pretest and posttest observations on both 
cognitive and affective instruments for both students and 
teachers . 

2. X 0 

This design was used in the evaluation of the Materials 
Acquisition component, 0 represented the teachers' 
responses to evaluative questions concerning the BMEP- 
produced curriculum after implementing the curriculum 
(X) in the classroom. 

3. 0^ X 0^ 

This design was used in the evaluation of the Community 
Participation component. 0^^ and 0 represented, 
respectively, observations on the Parental Involvement 
Questionnaire before and after the implementation of the 
BMEP (X) . 

The major evaluation questions of interest: 

1. Were the standardized test performances of BMEP and 
comparison-group-children from different ethnic and 
language groups equivalent? 

2. Were the Curriculum-Referenced Test (CRT) performances 
of BMEP and comparison-group children of different 
ethnic groups equivalent? 

3. What were the second-language achievement gains of 
BMEP students? 

4. What were the BNIEP teachers' self -reported usage rates 
and quality ratings of the BMEP-produced curriculum 
materials? 

5. To what extent were BMEP teachers successful in completing 
the requirements of the Master of Liberal Arts program 

in Bilingual Education at Southern Methodist University? 

6. What was the extent of success of the BMEP with respect 
to educating parents about the Program schools and 
community? 
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Evaluation Results; 



1. Anglo children in the comparison schools scored higher 
on the standardized tests than children in the BMEP. 
This difference was attributed to the higher socio- 
economic level of the comparison-group children and not 
to a weakness in the BMEP. 

2. When administered the standardized tests of basic 
concepts in English, the Spanish-dominant Mexican -American 
BMEP first- and second-grade children tended to score 
lower than English-dominant BMEP and comparison-group 
children. However, the lack of differences between Spanish- 
dominant BMEP children and other BMEP and comparison-group 
children at the third-grade level (where Spanish- 
dominant BMEP children were becoming bilingual) suggested 
that the deficiencies observed in the first two grades 

wera best attributed to language problems rather than to 
an inadequate development of basic concept skills. This 
contention was supported by data from the Curriculum- 
Referenced Test which showed that, when tested in their 
dominant language, BMEP children performed at least. as 
well as children in the comparison schools. 

3. No differences on the posttest CRT scores were observed 
for BMEP children of differenct ethnic groups in grades 
of kindergarten, one, and two. However, grade-three 
Mexican-American BMEP children outperformed Negro BMEP 
children on this measure. This result was not 
surprising, considering that the BMEP teachers felt that 
the curriculum was more appropriate for the Mexican- 
American child than it was for the Negro child. 

4. Large and statistically significant pretest-to-posttest 
gains were made on the English-as-a-Second-Language (ESL) 
Test by Spanish-dominant Mexican-American students. The 
gains made on the Spanish-as-a-Second Language (SSL) 
Test by English-dominant children were also significant 
but were considerably smaller than the ESL gains. 
Consistent with the results from the 1971-72 evaluation, 
it is apparent that Spanish-dominant Mexican-American 
children are becoming bilingual much faster than English- 
dominant Anglo and Negro students. 

5. Anonymous responses from teachers showed that the BMEP 
teachers did not exclusively use the BMEP-produced 
curriculum materials and that there were large differences 
in curriculum usage between subject areas. On a three- 
point scale, the majority of teachers rated the quality 

of all curriculum areas as "average". Also, a positive 
relationship was found to exist between the teachers' 
reported usage and quality ratings of the curriculum. 
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6. During the school year, most BMEP teachers, curriculum 
writers, and supervisors enrolled in four courses in 
the Master of Liberal Arts (MLA) in the Bilingual Edu- 
cation program, which resulted in BMEP personnel 
enrolling for 957 hours of course work. Only 

5% of their course work was not completed by June, 
1973. This was a considerably lower incompletion rate 
than is usually found in graduate school. In addition 
to the high course-completion rate, substantial pretest- 
to-posttest gains were observed for all MLA courses. 

7. Similar to the results found in the 1971-72 evaluation, 
the 1972-73 evaluation showed that the BMEP was 
successful in educating the parents about the Bilingual 
Program. The success was attributed to the Meritnda 
meetings , teacher-parent communications , principal- 
parent communications, and the visiting teachers' 
activities. The BMEP publication ^ Que Tal?*' was 
also found to be a useful information-dissemination 
tool. 



Recommendations of Evaluator : 

1. It is recommended that Spanish-dominant Mexican-American 
children be tested only in Spanish. It is further 
recommended that, where Spanish tests are not available, 
the Spanish-dominant children should not be tested. It 
is felt that the possible psychological damage to the 
child caused by attempting to respond to the test items 
in an unfamiliar language outweighs the value of the 
resulting (and probably invalid) data. 

2. More information concerning Negro history and culture 
should be included in the curriculum. This could be 
facilitated by either hiring additional Negro curriculum 
writers or by hiring individuals who are interested in 
researching and writing in the area of Negro history and 
culture . 

3. Project administrators should reevaluate the goal of 
bilingualism for all BMEP students. Perhaps a more 
realistic goal would be bilingualism for 1) all Spanish- 
dominant Mexican-American BMEP students and 2) those 
English-dominant BMEP students who demonstrate superior 
language skills. More time could then be spent perfecting 
the English language skills of English-dominant BMEP 
children who have shown difficulty with acquiring English 
language skills. 

The teachers' quality ratings of the BMEP-produced 
curriculum should be used by project administrators in 
the allocation of curriculum-writing talent. While no 
causal relationship was established between teachers' 
usage rates and quality ratings, there was sufficient 
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evidence to hypothesize that a curriculum which was uniform 
with respect to quality across subject areas would lead to 
uniform implementation across subject areas. It is also 
recommended that the teachers' implementation of the 
curriculum be more closely monitored by the resource teachers. 
The resource teachers should be required to submit 
curriculum implementation reports for each teacher to program 
administrators and evaluators. 

5. The Master of Arts (MLA) in Bilingual Education program 
at Southern Methodist University should be continued. 
During the 1973-74 school year, teachers new to the BMEP 
will take one MLA course per semester. Teachers in the 
second year of the MLA program will take two courses per 
semester. It is recommended that one of the courses for the 
second-year MLA students be taken on Saturday morning. 

This will decrease the very heavy workload experienced by 
these teachers when both courses were offered on weekday 
afternoons. 

6. Because of the vertical expansion within the BMEP from 
grades of kindergarten through three to grades of kinder- 
garten through four, it is recommended that an additional 
visiting teacher be added to the BMEP staff. Also, a 
second "El Quiosco" (the media production and social 
services referral center) should be established at Gabe 
P. Allen Elementary School so that BMEP parents in West 
Dallas can more easily participate in, and be served by, 
this component of the BMEP. 

/ 

Additional Information : 

73-142 Evaluation Report for the Bilingual-Multicultural Education 
Program, 1972-73, Wayne R. Murray, 97 pp. 

72-82 Bilingual-Multicultural Education Program Evaluation Report, 
1971-72, Daniel J. Macy and Wayne Murray, 75 pp. 
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A COMPARATIVE ANALYSIS OF TITLE I 
READING PROGRAMS 



Objectives of the Program : The six Title I programs that are examined 
in this study are:. 1) Hoffman Information System Reading Program 
(Hoffman), 2) Behavioral Research Laboratories/Sullivan Reading Program 
(BRL/ Sullivan) , 3) Southwest Regional Laboratory Reading Program (SWRL) , 
4) Listen, Look, and Learn Reading Program (LLL) , 5) Bilingual-Multicul- 
tural Education Program (Bilingual) , and 6) the Multiage Classes Program 
(Multiage). These six Title I programs have a variety of program objec- 
tives relating to specific parts of the various curriculum. However, each 
program does have a reading component and a specific objective, that in 
different ways attempts to raise the reading levels of the students enroll- 
ed in the six Title I programs. 

Purpose of the Evaluation : The evaluation of the six Title I programs was 
designed to determine and compare the effect of each program on the reading 
and language achievement of the students enrolled. 

Sample : A 15% sample was drawn from the population of students enrolled 
in each of the six programs under investigation. The only restrictions placed 
on the data being that students included in the sample must have both pretest 
and posttest achievement scores. 

Evaluation Design : The figures below schematically represent the evaluation 
design used in this study. 



1st Grade 




2nd Grade 



Same as on 1st Grade 



3rd Grade 




4th Grade 
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where X^^ represents students in the Hoffman reading program, X2 represents 
students in the BRL/ Sullivan reading program , X^ represents students in the 
SWRL reading program, X^ represents students in the LLL reading program, X^ 
represents students in the Bilingual program, and X^ represents students in 
the Multiage program. Variables defining the observation points (0) are 
the pretest and posttest standardized achievement tests. The major evaluation 
question of interest was: What are the effects of the six Title I programs 
on student reading and language achievement? 

Evaluation Results : In general, students in the Hoffman, BRL/Sullivan and 
SWRL programs showed higher adjusted mean scores^ for both reading and 
language totals, on a standardized achievement test than students in the 
other programs. At grade-level one,, students in the BRL/Sullivan program 
showed the highest level of performance on both the language and reading 
totals. At grade-level two, students in the SWRL program showed the highest 
level on reading while students in the Bilingual 2 program showed the highest 
levels on language. At grade-level three, students in the BRL/Sullivan pro- 
gram showed the highest level on both reading and language total. At 
grade four, no difference was found for students on reading total, however, 
BRL/Sullivan students showed the highest level of performance on language 
totals. In addition, the Hoffman program appears to show consistently 
good results for all four grade levels investigated. 



Recommendations of Evaluator : No recommendations specific to this report 
will be made, since this report is simply a comparison of Title I programs. 
Specific recommendations within each program have been made and can be 
found in the following Research Reports of the Department of Research and 
Evaluation, for Hoffman, BRL/Sullivan, SWRL, and LLL, see Research Report 
No. 73-185; for Bilingual-Multicultural Education, see Research Report 
No. .73-158; and for Multiage Classes see Research Report No. 73-142. 



Additional Information ; 

73-201 Comparative Analysis of Title I Reading 

Programs, 1973, Robert J. Costello 
and James P. Papay, 26 pp. 
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TITLE I MULTICULTURAL SOCIAL SCIENCE PROJECT 

Objectives of the Program : The 1972-73 Multicultural Social Science 
Project (MSSP) consisted of two related classes of objectives. The first 
was to field-test both the Multicultural Social Studies curriculum materials 
developed by the Southwest Educational Development Laboratory (SEDL) in 
grades one, two, and three, and those written by DISD MSSP curriculum 
writers in grades kindergarten, four, five, and six. The realization 
of this first objective included the actual writing of the curriculum 
materials and corresponding criterion-referenced tests by DISD MSSP 
curriculum personnel in accordance with a model SEDL format. The second 
class of objectives dealt with the validation of the MSSP curriculum 
through the assessment of the classroom performance of participating 
children assigned the Multicultural curriculum in grades kindergarten 
through six with respect to 1) curriculum mastery, 2) general social 
studies knowledge, 3) self -concept , and 4) school attitude. 

Purpose of the Evaluation : The evaluation was conducted to determine the 
extent to which the 1972-73 MSSP was able to realize its stated objectives 
and to provide information potentially useful to MSSP supervisory personnel 
and curriculum writers in their planning of the 1973-74 MSSP, 

Sample : Children attending the following five elementary schools were 
assigned the Multicultural curriculum during the 1972-73 school year in the 
grades indicated: Paul L. Dunbar and O. M. Roberts (kindergarten through 
six); N. W. Harllee and James Fannin (one through three); and David 
Crockett (one through two). In addition, children in grades kindergarten 
through six attending Stephen F. Austin and B. T. Washington Elementary 
Schools served as controls. 



Evaluation Design ; The two basic designs incorporated into the evaluation 
were as follows: 

a) 0^, 0 0^, where each 0^ represents an observation on a 

selected variable during the 1972-73 school year, and 

^1 ^2' ^^^^^ ^2 ^^P^^®^^^® ^ student performance measure 
functioning as a dependent variable, X represents the assignment of 
either the SEDL or DISD Multicultural curriculum within a given grade 
level, 0^ represents children's characteristics or pretest performance 
measures used as statistical controls, and the dashed line ( ) repre- 
sents the fact that the experimental treatment effect (X) was not assigned 
randomly over all schools. The specific evaluation questions investigated 
were as follows: 
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1. Did the children in each grade learn the MSSP curriculum material? 

2. What characteristics of children were related to success in learn- 
ing the MSSP material in each grade? 

3. Could the children's learning of the MSSP curriculum content in 
each grade be attributed to the MSSP instructional materials? 

4. Were the MSSP curriculum materials in each grade properly 
implemented? 

5. Was the general social studies knowledge of children assigned 

the MSSP curriculum in each grade greater than that of comparable 
children receiving traditional instruction? 

6. Did the children assigned the IISSP curriculum in each grade hold 
a more positive opinion of their own school learning skills com- 
pared to similar children receiving traditional instruction? 

7. Did the children assigned the MSSP curriculum in each grade 
display a more positive attitude toward classroom learning and 
school than similar children receiving traditional instruction? 

8. What were the opinions of participating classroom teachers re- 
garding selected aspects of the MSSP curriculum in each grade? 

Evaluation Results : 

1. Children's mean percentage of questions correct over all criterion- 
referenced unit mastery tests for the SEDL curriculum in grades 
one, two, and three were 86.0, 83.4, and 78.1, respectively. 
Children assigned the DISD Multicultural materials in grades 
kindergarten, four, five, and six obtained a mean percentage of 
79.6, 81.3, 78.2, and 68.2, respectively, over all unit tests 
attempted. However, although the kindergarten unit tests were 
criterion-referenced, the demands of writing the curriculum units 
in grades four, five, and six allowed sufficient time only for the 
preparation of informal unit tests over selected curriculum topics • 
The obtained unit test performance in grades four, five, and six, 
therefore, should be considered as indicators of learning rather 
than of curriculum mastery. 

2, Children's characteristics found significantly related to mean unit- 
test performance in each grade of the SEDL curriculum were initial 
SEDL curriculum knowledge, as measured by a criterion-referenced pre- 
test, and school attendance. The analysis of the unit performance 
data in grades one, two, and three also found that white, black, 

and chicano children realized similar high degrees of success on 
the SEDL curriculum. For the DISD curriculum materials, only initial 
general social studies knowledge in grades five and six was found 
related to performance on the unit tests over all grades. Because 
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the enrollment of both schools in which the DISD curriculum 
was field-tested was predominantly black, the effect of 
ethnicity upon performance of the Multicultural materials 
could not be investigated. 

3. Significant differences between the scores of SEDL and control 
children on criterion-referenced posttests indicated that both 
the grade-one and the grade-three curriculum produced children's 
learning. The lack of a signlticant difference between SEDL 
and control children on a similai test in grade two, however, 
suggested the possibility that the mastery of these children 
might have resulted from other sources of learning. Significant 
differences obtained between the performance of Multicultural 
and control children in grades kindergarten, four, five, and 
six also served to indicate that the curriculum in each of these 
grades produced children's learning, 

4. Teacher responses to a questionnaire based upon the results of 
classroom process evaluation by Title I Evaluation personnel 
indicated that the implementation of the SEDL and DISD curriculum 
components was generally satisfactory. However, teachers also 
indicated that they were not able to obtain all of the help they 
desired from MSSP staff and that they often did not receive all 
of the materials required each day to teach the curriculum. This 
latter defect was most prominent in grades four and five, and 
especially in grade six, in which the curriculum materials were 
all being written during the school year. Although more curri- 
culum units were written in grades kindergarten, four, five and 
six over the school yeat, due to a late start, only three of 11 
planned units were field-tested in kindergarten, three of five 

in grade four, three of seven in grade five, and one of seven 
in grade six. 

5. Significantly greater social studies achievement in favor of the 
Multicultural children as measured by standardized tests was found 
only for the SEDL material in grade three and for the DISD material 
in grade four. In the remaining grades. Multicultural and control 
children displayed the same degree of general social studies 
knowledge at the end of the school year. 

Children assigned the DISD Multicultural curriculum in grade four 
were found to hold more positive academic self-concepts as assessed 
by a verbal scale developed by Instructional Objectives Exchange. 
No other significant differences were found in children's self- 
concepts over grades kindergarten through six. 

Children assigned the DISD Multicultural curriculum in grade four 
also were found to have a more positive attitude toward school as 
measured by a verbal scale developed by Instructional Objectives 
Exchange. However, the control children in grade six displayed a 
significantly more positive attitude toward school than the Multi- 
cultural children. No other significant differences were found 
between experimental and control children in [];rades kindergarten 
through six. 
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8. Teachers in grades kindergarten through five generally indi- 
cated that the difficulty level of the material was appropriate 
for from 60 — 70% of participating children, with teachers in 
grades one and two tending to indicate the material was somewhat 
easy, and teachers in grades three, four, and five indicating the 
material was too difficult for approximately 25% of children in 
each grade. Kindergarten teachers felt the material was appro- 
priate for virtually all children in their classes. In this 
context, teachers recommended that 5.0, 17.3, 22.9, 25.4, 36.6, 
and 38.0% of the curriculum be revised in grades kindergarten 
through five, respectively. In grade six, teachers reported 
that the material was too difficult for 82.5% of their children 
and that they were forced to make classroom changes for over 82.5 
and 67.5% of the material to insure children's interest and learn- 
ing, respectively, during field-testing. Accordingly, grade- 
six teachers recommended that 87.5% of the grade-six curriculum 
be revised. With the exception of those in grade six, the large 
majority of teachers evidenced a positive attitude toward the 
Multicultural curriculum by indicating that they would like to 
teach in^ the program during the coming school year. 

Recommendations of Evaluator ; Based up.on the results of the 1972-73 MSSP 
evaluation, the following recommendations are offered: 

1. MSSP pp.rscnnel should complete whatever revision is required for 
the SEDL curriculum in grades one and three so that it can become 
available for implementation. For the grade-two SEDL curriculum, 
MSSP personnel should determine whether children are, in fact, 
learning from the curriculum materials or whether the curriculum 
is attempting to teach children that which they already know. 
This question can be answered by administering, on an experimental 
basis, the criterion-referenced unit mastery tests to experimental 
and control children as both a pretest and a posttest. Any further 
decis:-. 'jr. regarding implementation of the grade-two materials 
should await this or equivalent performance data. 

2. MSSP personnel should complete writing, revising, and field-testing 
all of the units planned in grades kindergarten, four, five, and 
six, perhaps placing greatest priority upon the grade-four and 
kindergarten materials that both show the greatest promise and 
provide direct upward and downward extensions of the SEDL curri- 
culum in grades one through three. It is essential that an 
integral part of this activity include that curriculum writers 
follow the model SEDL lesson format. In addition, MSSP supervisory 
personnel should direct curriculum writers to plan and construct 
technically adequate criterion-referenced unit tests that can 
function to assess curriculum mastery by participating children. 
This latter step is crucial since curriculum effectiveness can- 
not be considered validated without the demonstration of- learning 
mastery by participating children. In general, given the stated 
objectives of the MSSP, the writing, the field-testing, the 
necessary revision, and the eventual validation through student 
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mastery of all units within a grade level should properly pre- 
cede any implementation of the DISD Multicultural material. 
Finally, if it is the purpose of the HSSP to validate the 
curriculum materials being developed in grades kindergarten, 
four, five, and six for a population of white, black, and chicano 
children, then these curricula should at some point be tested 
in other than predominantly black schools. 



Additional Information : 

72- 98 Evaluation Design for the Multicultural Social Science 

Project, 1972-73, Michael R. Vitale, 32 pp. 

73- 141 Evaluation Report: Multicultural Social Science 

Project , 1972-73, Michael R. Vitale, 81 pp. 
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TITLE I CHILD DEVELOPMENT 
SPECIALIST PROJECT 



Objectives of the Program ; The Child Development Specialist Project 
(CDSP) consisted of the assignment of a certified elementary school coun-- 
selor to each of the 20 DISD elementary schools having the largest pro- 
portion of economically disadvantaged chil^r:en. The CDSP was funded 
through the Elementary and Secondary Education Act Title I program at a 
cost of $214,000. The CDSP objectives were to identify and help eliminate 
the school problems evidenced by disadvantaged children that ordinarily 
reduce their success in school achievement, to assist with the implementa- 
tion of the DISD group-testing program and provide training workshops 
designed to help teachers properly utilize the standardized test results, 
and to complete a number of "mini-experiments" that would demonstrate 
clearly the effectiveness of selected change-techniques in eliminating 
selected categories of children's problems. The activities of specialists 
(e.g., counselors) most relevant to these objectives included individual 
and group counseling with children and consultation with teachers, other 
school personnel, and parents. 

Purpose of the Evaluation ; The CDSP evaluation was designed to assess the 
general pattern of specialist activities during the school year, to deter- 
mine whether the CDSP objectives were realized, and to provide recommen- 
dations that would increase the future effectiveness of the Project. 

Sample t Three distinct samples supplied the information used in completing 
the evaluation. The first consisted of the specialists themselves who 
summarized both their own activity patterns as well as specific aspects 
of children's long-term counseling problems. The second consisted of 
principals and a random sample of teachers within each grade-level obtained 
from the CDSP and from 18 equivalent control schools who provided infor- 
mation relating to the nature and seriousness of children's classroom 
problems and to specialist effectiveness in helping teachers use DISD 
standardized test results. The third consisted of all children in grades 
one, two, three, and four CDSP and four equivalent control schools who 
provided information regarding DISD standardized test achievement. 

Evaluation Design : The components of the evaluation design corresponding 
to the comparisons planned within the different samples are presented 
schematically below: 

a) 0^ 0^ • • • 0 where each 0 represents an observation by a 
^ specialist during the school year. 
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b) R 0^ X 0^ 

R 0^ O2 where R represents the random selection of teachers 
within each grade level in CDSP and control schools, X represents the 
presence of a specialist in the CDSP schools, 0^ represents the responses 

of teachers at the end of the school year, the dashed line ( ) repre^ 

sents the nonrandom assignment of the CDSP treatment among the schools, 
and 0^ represents observations on concomitant variables available at the 
beginning of the school year that functioned as statistical controls. The 
design components used with the samples of principals and children were 
equi ralent to that shown above in b) except for the deletion of the R's 
to indicate that all principals and all children in grades one through 
four were included from their respective schools* The major evaluation 
questions investigated were the following: 



1. How much time did specialists devote to different kinds of 
activities in their schools, and what kinds of children's pro- 
blems did they face? 

2. How successful were specialists in eliminating children's pro- 
blems and what specialist activities contributed most to that 
success? 

3. Did children in CDSP schools evidence fewar classroom problems 
and greater school achievement than those attending comparable 
control schools? 

4. Did specialists help prepare teachers to interpret DI^^D standard- 
ized test results? 

5. What were the opinions of principals and teachers regarding 
selected aspects of the CDSP? 

6. Were specialists able to describe and validate as effective 
specific change-techniques applicable to specific children's 
problems? 



Evaluation Results : 

1. Specialists as a group spent over 50% of their overall activity 
counseling students and consulting with teachers about children's 
problems. Specialists also reported that approximately 50% of 
the children with whom they worked on a continuing basis 
evidenced classroom discipline and/or social problems. Pro- 
blems categories specialists faced less frequently included 
motivation, learning remediation, and family* 
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2. Specialists averaged a 63% degree of success in the elimination 
of children's long-term problems over grades K through six, with 
specialists being slightly less successful with children in grades 
two, three, and four. The results also showed that parental contact 
was associated with increased counseling success. However, length 
of the counseling episode, number of counseling contacts, and 
hours of teacher contact all were discovered to be related in- 
versely to counseling success. This latter finding was inter- 
preted to reflect the fact that more serious problems require 

more counseling activity while offering less opportunity for 
success. 

3. Children in CDSP schools generally were found to display about 
the same degree of classroom problems as comparable children in 
control schools, according to principal and teacher estimates. 
The one exception showed that the control children evidenced 
fewer learning problems. On DISD standardized tests, second- 
grade CDSP children performing significantly better than controls. 
However, no difference in standardized test achievement was 
found between CDSP and control schools in grades one, three, and 
four. 

4. Teachers in CDSP schools indicated that they obtained satis- 
factory help from specialists in the use and interpretation 

of DISD test results when such assistance was required. However, 
approximately 50% of all teachers obtained at least part of such 
help from specialists informally rather than in formal teacher 
workshops. About 33% of all CDSP teachers surveyed required 
no help from specialists in using DISD test results. 

5. Both teachers and principals in CDSP schools indicated that 
specialists seemed effective in reducing classroom problems 

and that specialists made a small, but significant, contribution 
to children's classroom learning. However, many teachers and 
principals (although not a majority) felt that specialists 
should devote a greater amount of time to helping solve class-- 
rooua problems . 

6. The CDSP objective calling for specialists to identify and validate 
specific change-techniques was not implemented during the school 
year. One possible reason suggested for this outcome was that 

the CDSP represented only a small part of the Project director's 
overall responsibility within the DISD. 

Recommendations of Evaluator ; Based upon the conclusions drawn from the 
1972-73 CDSP evaluation, the following recommendations are offered: 

1. The 1973-74 CDSP should focus upon the elimination of the specific 
classroom problems evidenced by Title I children in the areas of 
discipline, social adjustment, and motivation for learning* This 
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certainly seems to be an important need within Title I schools 
that most specialists should be well able to satisfy on a con- 
tinuing basis. 

2. The 1973-74 CDSP should identify and validate specific change- 
techniques that are effective in eliminating children's problems 
in each of the three areas listed above. If such specific tech- 
niques can be identified and disseminated, then the impact of 
the CDSP can be multiplied throughout the DISD. 

3. The 1973-74 CDSP should determine the most effective role for 
specialists in providing the assistance teachers require in 
interpreting and using DISD standardized test results. If, 
as the present evaluation showed, 33% of teachers require no 
help in using DISD test results, and at least 50% of teachers 
who need help obtain much of that assistance through informal 
specialist contact, then there seems little justification for 
the CDSP objectives to emphasize formal teacher workshops. 



4. The 1973-74 CDSP should have a full-time director who works 
closely with and coordinates the activity of specialists in 
realizing the objectives of the Project as a whole. Under the 
present administrative system, the CDSP consists of little more 
than elementary counselors working relatively independently 
within their own school, united by a common funding source. 
Although the present specialists do seem to perform an important 
service within their schools, such individual efforts do not 
necessarily imply that any goals of the CDSP as a whole will 
be realized. 



Additional Information : 

73-146 Evaluation of the Child Development Specialist Project, Michael 
R. Vitale, 1973, 52 pp. 

72-115 Evaluation Design for Child Development Specialist Project, 1972-73, 
Michael R. Vitale, 77 pp. 
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PAUL L. DUNBAR READING PROGRAM: 1972-73 



The evaluation of the Dunbar Reading Program for 1972-73 considered the 
educational outcomes of three primary reading programs which were imple- 
mented at the Paul L. Dunbar Community Learning Center. These programs 
included: (1) the Communication Skills Program, (2) the BRL/Sullivan 
Program, and (3) the Chandler Reading Program. 

Objectives of the Program ; The objectives for each of these programs 
were basically similar. Each of these programs emphasized development 
of reading vocabulary with particular attention toward word-attack 
skills, phonics, and the ability to sound out new words of different 
construction. Each program had its own unique curriculum. However, 
each program also had the samie goals, which were oriented toward the 
enhancement of the student *s learning capabilities through specialized 
instruction. 



Purpose of the Evaluation : The main purpose of the evaluation was to 
determine the differences, if any, between the students in the Dunbar 
Reading Program and students in comparison groups (different reading 
programs) by examining the standardized test results. Within each grade 
level, means, standard deviations, and gain-score comparisons were used 
to determine program differences. However, without the availability of 
appropriate controls uncontaminated by other programs within the Dunbar 
Learning Center, it was impossible to evaluate the complete effective- 
ness of each reading component. 

Sample and Design : The sample for the evaluation came from each of 
six grades at Paul L. Dunbar Community Learning Center and the equiva- 
lent two classes from each of the four control schools: C. F. Carr^ 
H. S. Thompson, B. T. Washington, and 0. M. Roberts. The design used 
for the analysis of the Dunbar Reading Program can be construed as a 
pretest - posttest design, schematically represented as 

Oi O2 O3 X-j^ 0^ O5 
Oi O2 ^2 0^ O3 

where the observations (0-j^ O2 O3) consisted of pretests on the Metro- 
politan Readiness Tests (MRT) , California Achievement Tests, (CAT) and 
the Comprehensive Tests of Basic Skills (CTBS) ; was the Dunbar Read- 
ing Treatment, X2 was the control Targeted Achievement in Reading Pro- 
gram (TARP) subcomponent reading treatment; and the observations (0^, O5) 
were the posttest performance scores on the CAT and CTBS. 

The regression approach was used for the analysis. Within the regression 
analysis framework, the posttest CAT and CTBS tests served as the 
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criterion variables, the factor of participation was the predictor 
variable, and the pretest MRT, CAT, and CTBS tests served as covariates. 
Particular hypotheses were tested by generating a restricted model 
and testing the from the restricted model against the from the 
full model. The resulting F-test enabled the determination of whether 
or not the observed event (treatment) was different from chance occur- 
rence . 



Evaluation Results : The results or the evaluation were not, overall, 
consistent. The lack of controlled assignment of pupils to treatments, 
pupils receiving additional treatment, and, at times, different entry 
levels precluded precise interpretation. However, the following general 
trends were observed: 

1. Within grades one through four, the Dunbar programs produced 
higher exit level performances than the corresponding compar- 
ison groups. However, in grades two and three, the comparison 
groups evidenced lower entering performances than pupils in 
the experimental Dunbar Program. 

2. Within grades five and six, where the entry levels were simi- 
lar, the Dunbar Chandler Reading Program did not produce any 
higher exit-level performances. However, the Chandler Reading 
Program was not uniformly implemented. 

3. There was not a clearly consistent trend within either the 
Dunbar or comparison programs favoring performance on either 
the Reading or Language tests of the CTBS. 



Recommendation of the Evaluator : The lack of appropriate control 
students in grades one through four strongly suggests that the follow- 
ing recommendations be made: 1) that no evaluation of the Dunbar 
Reading Program be implemented next year without the selection of 
appropriate control students within Dunbar; 2) that the control stu- 
dents be randomly selected from the Dunbar population and not given 
any of the additional instruction received by the experimental students. 

An ancillary recommendation regarding specific program implementation 
is that the second-grade Transition program be continued next year with 
the consideration of extending the concepts to other grade-levels for 
smoother transfer from one program to another. 

Additional Information : 

72-116 Evaluation Design for Paul L. Dunbar Reading Program: 1972-73, 
James P. Papay, 18 pp. 
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73-148 Evaluation Ro.port for Paul L. Dunbar Reading Program: 19 72-73, 
James P. Papay, 24 pp. 



Er|c 41 



PAUL L. DUNBAR MATHEMATICS PROGRAMS 



Objectives of the Program ; The Mathematics Program at Paul Dunbar 
Elementary School for 1972-73 consisted of two different curricula for the 
primary and intermediate levels: (1) Developing Mathematical Processes 
(DMP) , used in kindergarten and grades one and two, and (2) Quality 
Education Development (QED) , used in grades three, four, five, and six. 
The objective of both the DMP and QED components of the Dunbar Mathe- 
matics Program was to insure that children achieved learning mastery of 
the curriculum content assigned them during the school year. 



Purpose of the Evaluation : The evaluation was conducted to determine the 
extent to which children mastered the DMP and QED curriculum objectives 
and whether DMP and QED increased children's general mathematics knowledge 
more than traditional instruction. 

Sample : Two kindergarten, four first-grade, and three second-grade classes 
at Dunbar were assigned DMP, while five third-grade, four fourth-grade, six 
fifth-grade, and five sixth-grade classes at Dunbar were assigned QED, Kin- 
dergarten through sixth-grade children attending Booker T, Washington 
Elementary School served as controls. 

Evaluation Design : The two basic evaluation designs used are presented 
schematically below: 

(a) 0^, 0^9' 0^, where each 0^ represents an observation 
of a student during the school year and 

(b) 0^ X O2 

, where 0^ represents children's performance on 

UISD standardized mathematics tests administered at the end 

of the school year, X represents the DMP or QED 

curriculum treatment, 0^ represents a measure of children's 

initial mathematics knowledge, and the dashed line ( ) 

represents the fact that the DMP and QED treatments were 

not assigned randomly to classrooms in Dunbar and Washington, 

The specific evaluation questions investigated were as follows: 

1, What was the level, and rate of mastery, of DMP and QED 
curriculum objectives? 

2. What student characteristics were related to level and 
rate of curriculum mastery within DMP and QED? 

3- Was the general mathematics knowledge of children 
assigned DMP and QED greater than that of comparable 
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control children after statistically equating all of 
the children on prior mathematics knowledge? 



Evaluation Results : 

1. DMP criterion-referenced test results indicated that 
the average performance of children was at the satis- 
factory progress level or above for 100% of the tests 
in levels one (kindergarten) and two (grade one) and 
for 73.3% of the tests in level three (grade two). 

In addition, DMP test results showed that the children 
mastered a mean of 88.2% of objectives attempted in 
level two, 63.5% of the objectives in level one, and 
47.5% of the objectives in level three. For QED, teacher 
records indicated that the average child completed 
and mastered 1.2 levels (of the total of six) of the 
QED material in grades four, five, and six. However, 
in grade three, in which the full QED program was not 
implemented, children completed only an estimated 
0.2 curriculum levels. 

2. In both DMP and QED, only children's prior knowledge 
of the curriculum content was consistently related 
positively to rate and mastery of the curriculiam mate- 
rials in kindergarten through grade six. 

3. DISD standardized test results revealed no significant 
difference between the general mathematics knowledge of 
children assigned either DMP or QED In grades one, 
three, four, five, and six compared to that of controls. 
However, the controls in grade two were found to have 
significantly greater mathematics knowledge than the 
second-grade children assigned the DMP level-three 
curriculum. 



Recommendations of Evaluator : Based upon the results of the 1972-73 
evaluation, the following recommendations are suggested: 

!• DMP curriculum personnel should monitor closely the 

performance of second-grade children on the level-three 
curriculum materials during the coming 1973-74 school 
year in order to determine their appropriateness for 
Title I children, 

2, QED curriculum personnel should take necessary steps to 
insure that the curriculum is implemented properly in 
grade three as it was in grades four, five, and six, so 
that its effectiveness can be assessed. In addition, 
possible techniques to motivate children to work more 
rapidly through the QED objectives should be investi- 
gated. 
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3. Both and QED curriculum personnel should formally 
identify the components assessed by DISD standardized 
mathematics tests that do and do not correspond to 
their curriculum objectives. The results of these 
curriculum analyses would allow for a more satisfactory 
interpretation of DMP and QED children's performance 
on the DISD standardized tests. 



Additional Information ; 

72- 117 Design for Paul L. Dunbar Mathematics Programs, 

1972-73, Michael R. Vitale, 13 pp. 

73- 159 Evaluation of the Paul L. Dunbar Mathematics Programs, 

1972-73, Michael R. Vitale, 29 pp. 
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PAUL L. DUNBAR SCIENCE PROGRAMS 



Objectives of the Programs ; During the 1972-73 school year, two science 
programs were pilot-tested in the first grade at the Paul L. Dunbar 
Learning Center. One of these programs, Science: A Process Approach - 
Level A, was developed by the American Association for the Advancement 
of Science (AAAS). The second, the Conceptual Approach to Teaching 
Science (CATS) was developed by the Paul L. Dunbar curriculum develop- 
ment staff. In addition to the first-grade class, the CATS program was 
pilot-tested in one Minimum Foundation Kindergarten class. 

The AAAS program was designed to teach the scientific processes of 
observing, space-time relations, classifying, using numbers, measuring, 
inferring, and communicating. The concept base was built around common 
knowledge content such as color, size, quantity, shape, etc. The 
program does not include the more traditional science content. The 
CATS program, in comparison, is designed to teach these same processes 
but through a more expanded, traditional science content approach. 

Purpose of the Evaluation ; The evaluation of the Dunbar Science pro- 
grams was designed to determine the relative effectiveness of the two 
programs in teaching scientific processes and concepts. In addition, 
the effectiveness of the CATS program in meeting its own stated objec- 
tives was examined. This information could be useful in making dissem- 
ination or discontiniaation decisions. 



Sample ; Three first-grade classes, one Minimum Foundation Kinder- 
garten class and the Kindergarten-age children in three multiage classes 
were included in the study. One first-grade class and Minimum Founda- 
tion Kindergarten class received the CATS program. One first-grade 
class received the AAAS program. Science; A Process Approach. One 
first-grade class and the kindergarten-age children in the three multi- 
age classes served as controls receiving only the regular elementary 
science program. 



Evaluation Design ; Essentially two designs were used in this study, 
the first for kindergarten-age children and the second for first-grade 
children. 

K Class Assignment 0 X-, 0 

0 0 

1st R(Subjects) R(Classes) 0 X. 0 

0 X2 0 
0 X3 0 
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In the kindergarten study, children were assigned to the Minimum 
Foundation Kindergarten Class and Multiage classes in accordance with 
District policy. In the first-grade study, students were randomly 
assigned to classes. One class each was randomly assigned to each 
of the three treatments. X-j^ is CATS; the elementary science pro- 
gram; and X3, the AAAS program. Variables observed at each observation 
point are the CATS program Pre/Post Test and Science Test of the Test 
of Basic Experience. In addition, the CATS participants received 
interim assessment on CATS objectives. The major evaluation questions 
of interest were: 

1. What is the level of performance on CATS objectives of 
participants in that program? 

2. What is the level of performance on CATS objectives of 
children not participating in that program? 

. 3. What is the relative level of performance of participants 
in the three programs on the TOBE Science Test? 



Evaluation Results : 

1. First-grade CATS participants met the 80 - 80 criterion 
(80% of the children mastering 80% of the objectives) on the 
Posttest and on all but two of the unit tests. In addition, 
80% of the students in AAAS and the control group answered 
no more than 47% of the items correctly on the Posttest. 

2. Kindergarten CATS participants met the 80 - 80 criterion on 
only one unit. 

3. No significant difference was observed among the two kinder- 
garten programs on the TOBE Science Test. In addition, no 
significant differences were found between the three science 
programs at the first-grade level on the TOBE Science Test. 



Recommendations of Evaluator : 

1. The effects of the CATS program in a multiage situation should 
b e examined • 

2. The CATS program should not be implemented in a straight 
kindergarten situation. 

3. The three first-grade classes should remain intact for a 
follow-up study on the second level of both CATS and Science: 
A Process Approach. 



ERLC 



4. Level I of the CATS program, with minor revisions, appears to 
be ready for^^isseinj.nation to other Title I first-grade classes. 



Additional Information : 

72- 104 Evaluation Design for the Paul L. Duubar Science Programs, 

1972-73, Allen W. Scott, 24 pp. 

73- 139 Evaluation Report: Paul L. Dunbar Science Programs 1972-73, 

Allen W. Scott, 28 pp. 
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DUNBAR EARLY CHILDHOOD EDUCAlION 



Objectives of the Program ; During the 1972-73 school year, two classes 
of four-year-old children were taught at Paul L. Dunbar. One of these 
classes received the Cognitive Program for Early Childhood, a Program 
developed by the Dunbar Development Staff, based on the works of David 
Weikart of Ypsilanti, Michigan. The second class received a combination 
of the Direct Instructional System Teaching and Remediation (DISTAR) 
and the Early Childhood Education Learning System developed by the 
Southwest Educational Development Laboratory (SEDL) . 

The Cognitive Program focuses its attention on the cognitive or intel- 
lectual growth of the child. The Program is directed toward teaching 
the processes that children use to learn relationships between things 
which are in their environment. This is accomplished through the use 
of abstract symbols representing the environment. The Program is 
derived from the ideas of Piaget, but other theories are also considered. 

The Early Childhood Education Learning System, developed by SEDL, 
emphasizes oral language development, concept formation, visual train- 
ing, auditory training, motor training, and development of problem- 
solving skills. Activities for each training area are sequenced and 
presented in accordance with the chlld^s level of development. Each 
activity is written with a behavioral objective that relates to a 
higher- order skill. 

The DISTAR program was developed by Science Research Associates (SRA) 
from the Bereiter-Engelmann program of the University of Illinois. The 
DISTAR language program is designed to teach basic langua^- concepts, 
focusing upon the language of instruction. The activities of the pro- 
gram are sequenced, first teaching the use of complete statements in 
identifying objects, and then teaching a variety of concepts, such as 
color, shape, relative size, and class name, that reside in familiar 
objects . 



Purpose of the Evaluation : This evaluation was designed to provide 
information about the relative effectiveness of the two programs being 
examined. In addition, general information which could lead to Pro- 
gram revision was sought. 

Samp le: Thirty-two Title I-eligible four-year-old children were 
recruited from the Dunbar community. These children were randomly 
assigned to the two programs being examined. 
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Evaluation Design ; 



R (Subjects) 0 Xj^ 0 

0 X2 0 

Thirty- two four-year-old children were randomly assigned to two classes 
(sixteen each). One class received the Cognitive Program (Xj^) , and the 
other class received a combination of SEDL and DISTAR (X2). At each 
observation point (0), the BoehmTest of Basic Concepts and the Engle- 
man Basic Concept Inventory were administered. In addition, criterion- 
referenced instruments were administered throughout the year. 

As an added dimension of the study, the current level of achievement 
of past participants in the three programs was examined in relationship 
to classmates who did not have the benefit of an early childhood 
education program. The major evaluation questions of interest were: 

1. Is there any difference in concept development between 
participants in the two programs? 

2. What is the level of objective mastery of participants 
in the two programs? 

3. What is the current level of achievement of past partici- 
pants in SEDL, xjISTAR, and the Cognitive Program in 
relation to their age-mates? 



Evaluation Results ; 

1. No significant difference was observed between participants 
in the Cognitive and SEDL/DISTAR programs on Boehm Test of 
Basic Concepts performance. The SEDL/DISTAR participants 
scored significantly higher than the Cognitive participants 
on the Engleman Basic Concept Inventory, but since this test 
was developed as a criterion instrument for DISTAR, the 
importance of this outcome was reduced. 

2. Participants in both programs experienced little difficulty 
in mastering the majority of the objectives of the program 
in which they were participating. 

3. In attempting to examine the current level of achievement 
of past participants in the three ECE programs, it was 
discovered that valid data on less than one-sixth of the 
students were available in the Institutional Research Data 
Base. With this very small and possibly biased sample, no 
valid comparisons could be made among the programs. 
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Reconanendatlons of the Evaluator t 

1. It is recommended that pilot-testing of both programs be 
continued • 

2. Efforts should be made to improve the Cognitive Program. 

3. Efforts to locate past participants in the Early Childhood 
programs should be continued so that more meaningful follow- 
up information can be provided. 



Additional Information : 

72- 102 Evaluation Design for Dunbar Early Childhood Education, 1972, 

• Allen W. Scott, 23 pp. 

73- 150 Evaluation Report: Dunbar Early Childhood Education, 1973, 

Allen W. Scott, 25 pp. 
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SPECIAL SWRL PROJECT 



Objectives of the Program ; The Southwest Regional Laboratory (SWRL) 
First-Year Communication Skills Program (FYCSP) was designed for use 
by kindergarten children. The Second-Year Communication Skills Program 
(SYCSP) was designed for use by first-grade children. In the past, the 
FYCSP has been implemented in Dallas in first-grade classes with the 
SYCSP to follow as FYCSP is completed. However, the SYCSP has a Trans- 
ition Supplement that was designed for first-grade children who did not 
have sufficient mastery of FYCSP objectives to enable them to partici- 
pate successfully in SYCSP. The objectives of FYCSP and the SYCSP 
Transition Supplement are essentially the same. The decision was made 
to implement tl:e SYCSP in two classes at Dunbar without FYCSP. In its 
place, the SYCSP Transition Supplement was implemented. It was hoped 
that, in this way, the entire SYCSP could be completed in the first 



Purpose of the Evaluation ; The purpose of this special project study 
was to determine if the SYCSP could be successfully implemented without 
prior implementation of FYCSP. A second purpose was to determine if 
exclusion of FYCSP and the subsequent acceleration would affect achieve- 
ment test scores in reading and language arts. 

Sample : Four self-contained first-grade classes and the first-grade- 
aged children in four multiage classes were included in the study. 

Evaluation Design : 



First-grade-aged children at Paul L. Dunbar were assigned to either 
multiage or first-grade classes in accordance with the guidelines set 
forth for the multiage program. The children assigned to first-grade 
classes were then randomly assigned to four classes. Two of these 
classes received the FYCSP followed by the SYCSP (X-^) . The other two 
classes received SYCSP with the Transition Supplement (^2) . The first- 
grade-aged children assigned to multiage classes served as controls 
(X^) . The first observation (0^^) was the Metropolitan Readiness Tests. 
The second observation (O2) was the California Achievement Tests. In 
addition. Unit Criterion Assessment data were collected on the children 
in the STOL Programs. The major evaluation questions of interest were: 

1. What is the level of unit criterion performance of first- 
grade children participating in SYCSP without prior partici- 
pation in FYCSP? 



grade. 



First Grade (R(Subjects) 




Assignmei^t 



Multiage 
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2. Is there any difference in standardized test performance among 
first-grade FYCSP participants, first-grade SYCSP participants, 
and first-grade-aged multiage-class children who did not 
participate in a SWRL Program? 



Evaluation Results t 

1. First-grade children receiving SYCSP without prior partici- 
pation in FYCSP met and surpassed criterion on every outcome 
of both the Transition Supplement and SYCSP • Further, with 
the exception of six children, all children in these classes 
completed SYCSP, while the classes that began with FYCSP were 
only able to complete half of the Second-Year Program, 

2. No difference was found among FYCSP participants, SYCSP 
participants? and the controls that could not be attributed 
to entry characteristics. 



Recommendations of Evaluator : 

1. The SYCSP with the Transition Supplement appears to be the 
most appropriate Program for most first-grade children. 

2. Efforts to individualize instruction with SWRL Programs 
should be continued. 

3. The overall success of SWRL, not only at Dunbar, but through- 
out TARP, would appear to justify a close examination of the 
new kindergarten through three SWRL Communications System. 

4. A careful study should be made to determine if the SWRL 
System can be successfully and economically implemented in 
a multiage setting. 



Additional Inf oinnation : 

72- 120 Evaluation Design for the Special SWRL Study, 1972-73, Allen W. 

Scott, 9 pp. 

73- 162 Evaluation Report: Special SWRL Project, 1973, Allen W. Scott, 

12 pp. 
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PAUL L. DUNBAR COMPREHENSIVE MUSIC PROGRAM 



Objectives of the Program : If the purpose of education is to free man to 
realize his full potential as an individual and as a member of society, 
then education must provide the mental, physical, and spiritual discipline 
necessary for man*s development. In keeping with this purpose, the Com- 
prehensive Music Program (CMP) was developed and has as its major objec- 
tives: 1) to provide activities at which one can succeed, 2) to create 
an atomophere in which one perceives one's individual worth, 3) to promote 
self-understanding, 4) to improve self-concept, and 5) to provide a back- 
ground of experience which will continue to enrich the life of each indi- 
vidual. 

The curriculum developed for the Comprehensive Music Program is based on 
the knowledge that the musical experience is an individual, experience. 
It is a means to self-understanding and self-expression. 

It is believed that music education should be aesthetic education. Basic 
to the child's understanding of the art of music is aural discrimination. 
Through exploration and success-oriented experiences, the child develops 
sensitivity to the expressive possibilities of duration, pitch, timbre, 
dynamics, form, and texture. He perceives and reacts to the fusion of 
these elements through composition, improvisation, movement, singing, 
playing, and listening. The total Program is one of comprehensive 
musicianship . 

The background experience provided by the music education Program will 
continue to enhance other areas of learning and to enrich the life of 
the individual whether one is a composer, performer, listener, or a com- 
bination of these three. 



Purpose of the Evaluation : The evaluation of the CMP is intended to serve 
two functions; first, it was designed to assist Program developers in 
making any necessary revision decision, and secondly, to inform upper- 
level decision-makers of the progress Being made in the development of 
the music Program* 

Sample ; The design test sample included the children and teacher in one 
kindergarten class. 

Evaluation Design : Volunteer (Teacher) 0^^ ^21 ^2 ^22 ^31 ^3 ^32 

The volunteer teacher taught the three units (X^ , X^, and X^) to her 
class. Before teaching each unit, she administered a unit preassessment 
(0-^, 0^ , and 0^ ) . At the completion of each unit, she administered a 
unit posiassessment (O.y* ^22* ^^32^' addition, lesson assessment 
data were collected at the completion of each lesson. A unit is composed 
of several lessons. The teacher also filled out a Teacher Checklist at 
the completion of each lesson. The following evaluation questions were 
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Investigated: 

!• What is the level of performance on the lesson objectives of 
the Program? 

2. What is the level of performance on the unit objectives of the 
Program? 

3. What is the reaction of the teacher to the Program? 



Evaluation Results ; 

1. Unit test performance was found to be lower than desired for 

units one and three. Much of the difficulty could be attributed 
to the overly complex assessment methods. 

2« Lesson assessment results were acceptable for units two and 
three. The same technical difficulties observed on the Unit 
Posttest for Unit I were revealed for lesson assessments in 
that unit. 

3. The reaction of the teacher to the Program was generally 
favorable. She was, however, critical of the evaluation 
procedures in Unit I. 



Recommendations of Evaluator ; 

1. The Program should be pilot-tested in not more than four Kinder- 
garten and grade-one multiage classes at Paul L. Dunbar during 
the 1973-74 school year. 

2. The first unit, along with the evaluation procedures, should be 
extensively revised. 

3. Units IV-VII should be design-tested prior to being implemented 
as part of the pilot test. 

4. The Hawiian Music program should be implemented simultaneously 
with CMP to determine which is more effective. 

5. Staff development and Program implementation should be initiated 
early enough in the school year to assure completion of both 
programs. 



Additional Information ; 

72- 103 Evaluation Design for the Comprehensive Music Progran: 

Design Test 1972-73, Allen W. Scott, 8 pp. 

73- 151 Evaluation Report: Comprehensive Music Program, 

1972-73, Allen W. Scott, 11 pp. 
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TALENT EDUCATION PROGRAM 



O biectives of the Program ; The Talent Education Program is an adaptation 
of the techniques and philosophy of Shinchi Suzuki. These techniques 
have been incorporated in a strings Program which was designed to meet 
the needs of children and their parents in the Dallas Independent School 
District. A major tenet of this philosophy contends that children, even 
very young children, can learn best from their parents. As a result, 
parent participation is a major part of the Program. The music learning 
system is designed to promote self-understanding, to improve self -concept, 
and to provide a background of experiences which will continue to enrich 
the life of each individual. The experiences are success-oriented and 
develop musical skills and concepts in such a way that they encourage 
reinvolvement in other areas of learning. Students in the Program will 
be expected 1) to demonstrate musical performance skills commensurate 
with individual potentiality; 2) to improve auditory discrimination 
through prescribed listening-response activities; 3) to demonstrate an 
increased understanding of the elements of music and their interrelation- 
ship; 4) to demonstrate an effective working relationship with the 
participating parent; and 5) to demonstrate improved self-concept after 
successful experiences in the music Program. 

Purpose of the Evaluation : The evaluation of the Talent Education 
Program was designed to serve three purposes. First, the evaluation was 
designed to provide infomation for upper-level decision-makers which 
could lead to decisions regarding dissemination, continued extended pilot- 
testing, or redirection of efforts. A second purpose was to provide 
information for the development staff which could lead to needed Program 
revisions. The third purpose was to identify valuable resource persons 
for future staff development. 

Sample : A stratified random sample of the pupils of nineteen Dallas 
elementary string teachers was included in the investigation. Stratifi- 
cation was based on the proportion of the children taught by the teacher 
placed in each category (level) . 

Evaluation Design : 

Program Identification Stratification R (subjects) 



x^o 
x.o 

4 
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The program of ^ach of the nineteen teachers was classified as Talent 
Education with parent participation (X^ , Talent Education without 
parent participation (^2^^ Dallas String Program (X^), or Paul Holland 
(X,). Five pupils were selected from each teacher's enrollment in one 
scnool by stratified random sampling. Stratification was based on the 
proportion of pupils in each category or level. Observation (0) consisted 
of video-taping the performance of each child individually without 
accompaniment. A panel of three judges then rated the performance of each 
child on the auditory variables of Tone Quality, Intonation, Phrasing, 
Articulation, Pulsation, and Overall Performance, without seeing the 
children or knowing which program or teacher they represented. The 
following evaluation questions of major interest were investigated: 

1. Is there any difference in string performance ratings 
among the four programs evaluated? 

2. Is there any difference in string performance ratings 
between pupils in Title I schools and that of pupils 
in non-Title I schools? 

3. What is the relationship between the length of time 
a pupil receives string instruction and the category 
in which he is placed for each or the four programs? 

4. Is there any difference in the standardized achievement 
test performance of children in Title I schools who 
participate in TEP and that of children in Title I 
schools who do not participate in TEP? 

Evaluation Results : 

1. Students in Talent Education Program, with parent partici- 
pation, received higher ratings, on all the variables 
considered, than students in the other three music programs. 
However, students in the Talent Education Program, without 
parent participation, received the lowest ratings. 

2. Children from Title I schools (all of whom were TEP 
participants with parent participation) received signifi- 
cantly higher ratings on all string performance variables 
than participants in all programs in non-Title I schools. 

3. TEP participants with parental involvement progressed at 

a significantly faster rate than participants in all other 
programs. TEP participants without parental involvement 
progressed at a significantly faster rate than participants 
in either the Paul Rolland or the Dallas String programs. 



ERLC 



4. TEP participants in Title I schools scored higher on 
standarized achievement tests than non-participants in 
the same schools. In general, differences at Dunbar 
were highly significant. Participants, on the average, 
were a full year ahead of non-participants in grade 
equivalency. 



Recommendations of Evaluator : 

1. The Talent Education Program should be given serious 
consideration for dissemination to any school with an 
existing string program if, and only if, parent par- 
ticipation can be reasonably assured. 

2. The Talent Education Program should be disseminated 
to other Title I schools on a tuition-free basis with 
a parental participation requirement. 

3. The Program should be expanded in the current Title 
I schools as much as possible- 

4. Efforts to refine the string performance evaluation 
procedures should be continued. 

5. Refinement and extension of TEP should continue at 
the Paul L. Dunbar Learning Center. 

6. A carefully designed longitudinal study should be 
conducted in the Title I expansion schools to 
determine whether TEP participation has a causative 
relationship with academic achieveraenit gains or 
merely a concomitant relationship. 



Additional Information : 

72- 101 Evaluation Design for Talent Education, 1972-73, Allen W. 

Scott, 16 pp. 

73- 149 Evaluation Report: Talent Education Program, 1972-73, Allen 

W. Scott, 61 pp. 
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FUNDAMENTAL LEVELS IN PHYSICAL SKILLS 



Objectives of the Program ; Test results and observation indicate 
that children first entering school are lacking basic concepts of 
movement as well as movement skills. The Fundamental Levels in 
Physical Skills (FLIPS) program compensates for these apparent de- 
ficiencies by presenting basic skill development activities that 
serve as a foundation for all physical development. The emphasis 
of the program is on individual development, thus there is an 
absence of team-oriented activities in the first level of the pro- 
gram. Children participating in the program should be able to 
demonstrate fundamental movements basic to all physical activities, 
a knowledge of body parts and their movements, a knowledge of direc- 
tional concepts, skills in rhythmic activities, and abilities in all 
major areas of physical development: balance, agility, strength, en- 
durance, and flexibility. 



Purpose of the Evaluation : The evaluation of Level I of the FLIPS 
program was designed to provide information to both program developers 
and District decision-makers. The information provided program devel- 
opers was designed to lead to further program refinement. The infor- 
mation provided decision-makers was designed to facilitate decisions 
regarding dissemination, extended pilot-testing, continued pilot- 
testing at Dunbar, or, in the event the program shows little promise, 
redirection or discontinuation of efforts. 



Sample ; Two self-contained first-grade classes and one kindergarten 
class were included in the study. The children in the two first-grade 
classes were randomly assigned to classes. All three classes partic- 
ipated in the FLIPS program, but one of the first-grade classes was 
taught by a physical education specialist, while the other first-grade 
class and the kindergarten class were taught by their homeroom teachers. 



Evaluation Design; 



K Assignment 0 X^^ 0 



0 ^ 
0 ^ 
0 X^^ 0 

0 x^5 0 
ox,, 0 
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1st R(Subjects) R(Classes) 




^11' ^12 ^16 ^^P^^^^^^ six units of the program taught by 

a teacner without a physical education specialization. ^21* ^22* 

X^^ X^g represent the six units of the program taught by a 

pnysical education specialist. The variable examined at each ob- 
servation point, 0, is the performance of the participants on the 
unit objectives. The major evaluation questions of interest were: 

1. What is the level of unit posttest performance of 
program participants? 

2. What is the gain from pretest to posttest on unit 
objectives for program participants? 

3. What is the difference in pupil performance between 
children taught the program by physical education 
specialist and that of children taught the program 
by nonspecialists? 



Evaluation Results ; The program was not implemented until January 
due to the multiplicity of programs being implemented by the program 
teachers. Conflicts in scheduling of released time for these teach- 
ers prevented implementation of the formal staff development that 
accompanies the program. In spite of these limitations, the follow- 
ing results were observed: 

1. At least 80% of the children mastered at least 80% 



of the objectives for every unit taught. 
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2. Since many of the objectives of the first level of 
the program were considered essential to future 
cognitivej, as well as psychomotor, development, no 
arbitrary increase criterion was established. 
Marked increases were observed on every unit of the 
program for all three classes. 

3. The unit pretests pointed out pupil deficiencies 
among first-grade children in areas that a first- 
grade child would be assumed to have prior mastery. 
These deficiencies were generally eliminated by the 
program. 

4. Although the children taught by the physical educa- 
tion specialist tended to have a slightly higher 
level of performance on the unit posttests, the 
children taught by the nonspecialists had no diffi- 
culty meeting the 80 - 80 criterion (80% of the ob- 
jectives being mastered by 80% of the participants). 



Recommendations of Evaluator ; 

1. Level I of the FLIPS program is recommended for ex- 
tended pilot-testing during the 1973-74 school year. 

2. Level I of the program should be tested in first- 
grade, kindergarten, and kindergarten and !irst- 
grade classes. 

3. Alternative activities for children demonstrating 
mastery of objecti\;es on the pretest should be given 
consideration. 

4. Preservice staff development and program implementa- 
tion should be initiated early enough to facilitate 
completion of the program by the end of the school 
year. 

5. A program pre-posttest based on the objectives of 
Level I of FLIPS should be developed to assess 
overall gain. 

Additional Information : 

72-108 Evaluation Design for Dunbar Physical Skills Program, 1972- 
73, Allen W. Scott, 25 pp. 



ERLC 



73-138 Evaluation Report: Fundamental Levels ia Physical Skills, l''";3, 
Allen W. Scott, 24. pp. 
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LEARNING THROUGH PIANO ; DALLAS PLAN 



Objectives of the Program ; The Learning Through Piano curriculum is a 
criterion-referenced, individually prescribed program of group piano 
instruction. The curriculum package is divided into four levels of 
instruction to be used with grades two through six in the elementary 
schools Included in each level is the teacher handbook, including lessons 
and tests, student materials, compositions written to supplement specific 
objectives, teacher resources, transparencies, and record forms* 

The curriculum is designed to allow each child repeated opportunities of 
success and is flexible enough in use of materials to give each child 
musical experiences relating to his own culture and cultures of others. 
Emphasis is placed on developing musical concepts and skills in such a 
way that other areas of learning are enhanced. The program is designed 
to: develop the concepts necessary to play the piatvo musically and with 
understanding, develop attitudes and values that will allow each child 
to make an intelligent decision about the place music will have in his 
life, build musical concepts and skills in such a way that will allow 
success and reinvolvement in other areas of learning, and provide the 
framework from which the student can organize music as a discipline. 

Purpose of the Evaluation ; The evaluation of Learning Through Piano - 
Dallas Plan was designed to serve two purposes. First, it was designed to 
aid upper-level decision-iiiakers in making decisions regarding dissemination, 
continued pilot-testing, or the redirection or discontinuation of efforts. 
The secondary purpose was to provide the development staff with infor- 
mation that would lead to program revision decisioxis. 

Sample : Eight elementary schools were included in the study. The four 
experimental schools were Bayles, Sudie Williams, Nathan Adams, and 
Paul L. Dunbar, while Everette L. DeGolyer, Stevens Park, Dan D. Rogers, 
and Ascher Silberstein made up the control schools. In addition, sixth- 
grade students at Paul L. Dunbar who were not LTP-DP participants served 
as controls for sixth-grade students in that school who were participants 
in the program. 

Evaluation Design ; 

Participation Selection 



Vo lun teers Part ic ipa t ion 
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The four participating schools were matched as nearly as possible with the 
four control schools. The fact that all eight schools had to be selected 
from the elementary schools offering class piano restricted tb?. choice • 
The four experimental schools implemented LTP-DP, designated by X^. The 
four control schools implemented the regular Dallas class piano program, 
designated by The observations, designated 0^, consisted of judges' 

ratings on piano performance and a written test over music knowledge and 
listening skills. 

The second part of the study compared sixth-grade children at Paul L. 
Dunbar who participated in the program (X^) with other sixth^grade 
children in the same school who did not participate in the program on 
(O2) reading achievement, language achievement, and arithmetic achieve- 
ment. The major evaluation questions of interest were: 

1. Wtiat is the comparison between LTP:DP participants and other 
DISD piano students on piano pertormance after two years of 
instruction? 

2. What is the comparison between LTP:DP participants and other 
DISD piano students on listening skills and cognitive aware- 
ness of the elements of music after two years of instruction? 

3. What is the effe'^.t of two years of participation in LTP:DP 
on standardized achievement test performance in a Title I 
school? 



Evaluation Results : 

1. LTP:DP participants received significantly higher ratings than 
the control subjects on eleven of the thirteen performance 
categories. No significant difference was observed on the 
other two categories. 

2. LTP:DP participants scored significantly higher than the 
control subjects on six of the eight subtests of the written 
test over listening skills and cognitive awareness of the 
elements of music. No significant differences were observed 
on the other two subtests. 

3. Sixth-grade LTP:DP participants at Paul L. Dunbar scored 
significantly higher than non-participating sixth-graders in 
the same school on the Reading Total, Language Total, and 
Arithmetic Total subtests of the Comprehensive Tests of Basic 
Skills. 
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Recommendations of Fvaluator; 



1. Learning Through Piano; Dallas Plan, Level IT., should be 
given serious consideration for District-wide dissemination. 

2. A study of the effect of varying types and intensities of staff 
development should be conducted during the 1973-74 school year. 

3. Dissemination of the program on a tuition-free basis to other 
Title I schools should be given serious consideration. 

4. A carefully designed longitudinal study of the effects of 
LTP:DP participation on standardized achievement test perform- 
mance should be conducted in the next Title I school(s) to 
which the program is disseminated. 



Additional Information ; 

72- 100 Evaluation Design for Learning Through Piano; Dallas Plan, 1972, 

Allen W. Scott, 23 pp. 

73- 137 Evaluation Report; Learning Through Piano: Dallao Plan, 1973, 

Allen W. Scott, 28 pp. 
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PAUL L. DUNBAR 
VISUAL ARTS PROGRAM PILOT TEST 



Objectives of the Pro pram ; The Dunbar Visual Arts Program is a compre- 
hensive, sequential Program based on the observed needs of Dallas inner-r- 
city children. The Program had undergone extensive design- testing prior 
to the 1972-73 pilot test. Some of taa major goals of the Program are 
to enable children to develop knowledge of the concepts, principles, and 
elements of art; develop skills with, tools and media; develop an aware-, 
ness of beauty, its sources and potential in their environment; create 
original art products; and achieve success in art which is enjoyable to 
them. 



Purpose of the Evaluation * The evaluation of the Dunbar Visual Arts 
Program was designed to determine whether the Program was appropriate 
for a Gelf-contained kindergarten class, a self-contained first-grade 
class and a kindergarten-through-one miltiage class. In addition, the 
effectiveness of each component of the Program was examined. The 
purpose of this examination was to aid in decision-making regarding 
future dissemination, extended pilot- testing , continued pilot— testing 
at Dunbar, or, in the event the Program shows little promise, re- 
direction or discontinuation of efforts. 



Sample : One kindergarten class, one kindergarten-thrcugh-one multiage 
class > and one first-grade class participated in the Program. The 
children in the first-grade class had been randomly assigned to this 
class. The other two classes were intact with no random assignment. 



Evaluation Design : First-grade R(Siibjec-j) assignment 0X0 

0 G 
0 X^ 0 
0 X^ 0 

Kindergarten assignment 0X0 

0 0 
0 X^ 0 
0 X^ 0 

Multiage assignment 0 X^ 0 

0 X2 0 
0 X3 0 
0 X^ 0 

One self-contained first-grade class, one self-contained kindergarten 
class, and one teain-taugh.t multiage class received the Dunbar Visual Arts 
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Program. Only the first-grade class was farmed by random assignment. X^^ 
^2' ^3' ^4 represent the four units of the Program that were imple- 
mented. A unit pretest and posttest were administered before and after 
each unit, respectively. These tests were designed as inferential measures 
of the unit with which they were associated. More direct objective 
measurements were made at the end of each lesson. The major evaluation 
questions of interest were: 

1 . What is the level of performance on unit tests of 
kindergarten participants, of first-grade participants, 
and of kindergarten-through-one multiage participants by 
age group? 

2. What is the level of pupil growth from unit pretest to 
posttest of kindergarten participants, first-grade 
participants, and kindergarten— through-one multiage 
participants by age group? 

3. What is the level of performance on lesson assessment s 
of kindergarten participants, first-grade participants, 
and kindergarten-through-one multiage participants? 

Evaluation Results : 

1. The 80-80 criterion (80% of the children correctly 
answering 80% of the items) on the unit tests was met by 
the first-grade and mulitage classes on 75% of the units. 
The kindergarten class, on the other hand, failed to 
meet criterion on any of the units. 

2. The validity of the unit tests as measures of the 
Program objectives was seriously questioned when they 
were found to yield markedly different results from 
those obtained by the more direct lesson assessments. 
This discrepancy has led to the extensive revision of 
the unit tests as well as the unit structure for the 
1973-74 school year. 

3. The lesson assessments revealed that the Program was 
quite effective in meeting its own objectives. These 
results were consistent for kindergarten, first grade, 

and kindergarten-through-one multiage classes. / 



RecoimnendatioTis of Evaluator : 

1. The Program appears to be ready for extended pilot- 
testing in a limited number of schools (no more than two 
or three schools). 

2. Additional efforts should be exerted in the development 
and validation of unit tests. 
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3. Kindergarten-thxough-iDultiage classes and self^-contained 
first--grade classes should be given first priority oyer 
seJ t-cQutained kindergarten classes as extended pilot-test 
si tps, 

4. Staff development and program implementation should be 
initiated early enough to facilitate completion of the 
Program during the scfiool year, 

5. The participants in Level I of the Program should be 
kept together for pilot-testing of Level II during the 
1973-74 school year. 



Additional Information : 

72- 99 Evaluation Design for Paul L. Dunbar Visual Arts Program, 1972, 

Allen W. Scott, 14 pp. 

73- 136 Evaluation Report; Paul L, Dunbar Visual Arts Program 

Pilot Test, l972-'73, Allen W, Scott. 27 pp. 
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CAREER DEVELOPMENT CENTER 
SKYLINE - CONTEXT EVALUATION 



Objectives of the Program : The Skyline Career Development Center provides 
extensive preparation in a specific area of opportunity without sacrificing 
college entrance requirements. Any high school student in the Dallas 
Independent School District who meets certain minimal requirements can 
enroll in any of a variety of programs. Thereby the Career Development 
Center becomes an extension of all District high schools that offers 
participants the opportunity to obtain knowledge and experience in tech- 
niques of an infinite variety in selected advanced study and career areas 
heretofore unavailable. The Center's innovative approach demands the best 
teaching staff that can be provided. This means not only exceptional 
academic and technical knowledge and the ability to impart it but also personal 
involvement above and beyond the ordinary. 

Purpose of the Evaluation : The context evaluation at the Career Develop- 
ment Center was to continue the data base established during the 1971-72 
school year. This data base allowed for comparative studies during the 
school year, as well as longitudinal research. The sCudent and instructor 
data bases were also needed to inform administrative decision-makers of 
the composition of the student body and instructional staff at the Career 
Development Center. Finally, all other areas of evaluation at the Career 
Development Center required an accurate accounting as to the student and 
instructor population on a day-to-day basis. 

Sample : All students who were accepted into the Career Development Center 
were involved in the data collection for context evaluation. This included 
returning students as well as new students. Also included were all 
instructional staff members during the 1972-73 school year. 

Evaluation Design : The context evaluation at the Career Development 
Center involved the development and maintenance of an in-depth file on 
each student and instructor at the Career Development Center. Major stu- 
dent variables included were name and identification niambers, ethnicity, 
parental data, enrollment information, and various achievement and 
aptitude test scores. Instructor variables dealt with previous experience, 
non-teaching experience, and educational background. The design also 
involved continuous updating with reports presented in November, 1972, 
January, 1973; and February, 1973. The major evaluation questions of 
interest were: 

1. What, if any, changes occurred in the student body and 
instructional staff between the 1971-72 and 1972-73 school 
years at the Career Development Center? 

2. \^niat, if any, changes took place during the school year 
at the Career Development Center? 

3. What was the extent of withdrawal, and what reasons were 
given for withdrawing, from the Career Development Center? 
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Evaluation Results : 



1. The final CDC enrollment at the completion of the 1972-7.3 
school year was 2^01 students; this represented an 
increase of 27.3% over the ip29 students enrolled in CDC 
at the end of the 1971-72 school year. 

2. The ethnic makeup of the Career Development Center was 
relatively unchanged except for an increase of 3% for 
Negroes and a decrease of 4% for Mexican-Americans. The 
final ethnic composition was 71% Anglo, 22% Negro, and 
7% Mexican-American. 

3. Thirteen clusters remained predominantly (over 90%) one 
sex in makeup. They were: Child Care, Cosmetology, 
Aeronautics, Architectural Drafting, Business, Construc- 
tion, Electronics , Fashion, Interior Design, Television 
Arts, Transportation Services, Environmental Control, 
and World of Manufacturing. 

4. Twelve clusters remained predominantly (over 80%) Anglo 
in ethnic makeup. They were: Aeronautics, Aesthetics, 
Architectural Draf ting , Drama , English , Horticulture, 
Mathematics, Music, Photography, Science, Television Arts, 
and Transportation Services. 

5. Approximately 77% of the instructors at the Career 
Development Center have college degrees. The average ' 
instructor at the Career Development Center has nine 
years' work experience in a field related to his 
teaching assignment. Those individuals having no college 
degree average 17.3 years of work experience in their 
teaching field. All teaching staff are certified in 
accordance with State regulations governing teachers in 
their respective fields. 

6. Approximately 67% of the student body at the Career 
Development Center were full-time Skyline High School 
students. This reflects no change from the 1971-72 
school year. 

7. Enrollment by cluster ranged from a low of 13 in 
Mathematics to a high of 166 in Business. Generally, the 
clusters with the larger enrollments consisted of 
several sub-components. 

8. Generally, the academic clusters drew students with 
strong academic backgrounds. 

9. The withdrawal rate at the Career Development Center 
during the 1972-73 school year was 11.9%. This compares 
favorably to the approximately 22% withdrawal rate 
during the 1971-72 school year. Approximately 50% of the 
withdrawals took place during the first two weeks of 
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school or at the semester break. The two major reasons 
for withdrawal were desiring to return to home school 
full-time and leaving the District, 



Recommendations of Evaluator ; 

!• Context evaluation should be continued to provide 

necessary inputs for administrative personnel, as well 
as data for other evaluation efforts. 

2. Given the extent of predominantly Anglo or unisex 
clusters, additional steps should be taken to encourage 
minority students or students of the opposite sex to 
enroll in those clusters. 

3. Considering the limited representation of minority 
groups on the instructional staff at the Career Develop- 
ment Center, efforts should be redoubled to hire 
qualified minority instructors. 



Ad di tional In f orma t i on : 



S72-73 - 8 Preliminary Student Context Data Report, 1973, Walter II. Hardi, 
66 pp * 

S 72-73 - 9 Preliminary Instructor Context Data Report, 1973, Walter 
J. Hardi, 8 pp. 

S72-73 - 13 Initial Student Context Data Report, 1973, Walter J. Hardi, 
71 pp . 

S 72-73 - 19 First-Semester Student Withdrawal Report at the Skyline 
Career Development Center, 1973, Walter J. Hardi, 8 pp. 

872-73 - 22 First-Semester Context Data, 1973, Walter J. Hardi, 66 pp. 

S72-73 - 29 Preliminary Graduate Student Report, 1973, Walter J. Hardi, 
14 pp. 
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SKYLINE CAREER DEVELOPMENT CENTER - PROCESS EVALUATION 



Objectives of the Program : The Skyline Career Development Center Is 
the product of the Dallas Independent School District's commitment to 
career education. The facilities and equipment are modem and repre- 
sentative of that found in various careers for which the students are 
receiving training. (There are twenty-eight clusters representing 
many different career areas.) 

Curriculum continues to be developed for the Center which may have 
impact throughout the District. The curriculum is to be unique to 
the District. It is also meant to be flexible, enabling students to 
perform at varying levels and terminate at different points in any of 
the offerings. 

Instructors were also encouraged to explore new relationships, try 
new ideas, and investigate innovative teaching strategies. This 
stemmed from the concept of the Career Development Center as a cat- 
alyst for positive change. 

Purpose of the Evaluation ; Process evaluation (derived from the CIPP 
model) was designed to detect or predict, during implementation stages, 
defects in the procedural design (including setting) or its implementa- 
tion. This provided needed periodic feedback to the project manager 
and other decision-makers, who had the responsibility for continuous 
control and refinement of plans and procedures. 

Sample ; All twenty-eight clusters at the Career Development Center 
were evaluated during the 1972-73 school year. 

Evaluation Design ; The overall strategy was to identify and continually 
monitor the potential sources of failure in the project. Personal 
interviews and observations combined with survey questionnaires pro- 
vided the information for this part of the evaluation. Observation 
and interview efforts were spread over two complete rounds from 
September, 1972, to April, 1973. The six major evaluation questions 
under process evaluation were; 

1. Were the facilities and equipment adequate for the intended 
activities; if not, what changes were recommended? 

2. To what extent was the program, as designed, being implemented? 

3. To what extent were the facilities and equipment being util- 
ized? 
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4. To what, degree was the curriculum, as implemented, customized 
to the needs of the individual student? 

5. Were instructors aware of the appropriate sources of assistance 
concerning any problems or questions they might have? 

6. How much time was spent by instructors on administrative work 
and was it worthwhile? 



Evaluation Results ; 

1. There existed, and, in some cases, still exists, a need for 
additional materials and equipment in a few clusters. 

2. Instructors universally held the opinion that practically all 
curriculum would need some form of revision. 

3. There was a general satisfaction on the part of everyone over 
the operation during the 1972-73 school year as compared to 
the 1971-72 school year. 

4. Most clusters were found to be individualizing the implementa- 
tion of the curriculum as written, and students seemed most 
enthusiastic to learn. 

5. Most instructors were aware of the appropriate channels for 
obtaining information or assistance at the Center. 

6. Of all administrative forms required of personnel at the Career 
Development Center, only two were not considered by instructors 
to be worth the time required to fill them out. They were the 
Curriculum Revision Form and a TEA Course Data Form. 



Recommejidations of the Evaluator ; 

1. Further efforts should be made by the Research and Evaluation 
staff to provide observational data concerning curriculum 
imp lemen t a t i on . 

2. Due to the sensitivity of the information obtained, care must 
be taken in the handling, dissemination, atid reaction of 
interim reports. 

3. While developmental projects lend themselves to constant 
change, additional attempts should be made to test existing 
curriculum before allowing changes to take place. 
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4. Given the amount of time required and the suggestions made by 
instructors, assistance in completion and/or alteration of 
administrative forms should take place over the summer of 19 73. 

5. Because of the nature of the information collected, it is sug- 
gested that C'msiderations enumerated within the text of this 
report, under Conclusions, be examined before reacting to the 
above findings and recommendations. 



Additional Information: 



72-105 Plan for the Evaluation of the Skyline Career Development 

Center 1972-73, Thomas G. Krueck, 21 pp. 

S72-73-11 Final Process Evaluation Report for Round 1, 1973, Janice 
Lave, Bob Clements, Dan Peterson and Thomas Krueck, 29 pp. 

S72-73-15 Instructors* Opinions toward Administrative Forms Used at 

Skyline Career Development Center, 1973, Janice Lave, 17 pp. 

S72-73-30 Final Process Evaluation Report for Round 2, 1973, Thomas 
Krueck and Dan Peterson, 17 pp. 



Also Interim Process Evaluation Reports: 



S72-73-1 
S72-73-2 
S72-73-3 
S72-73-4 
S72-73-5 
S72-73-6 



S72-73-18 
S72-73-20 
S72-73-24 
S72-73-25 
S72-73-26 
S72-73-28 
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SKYLINE CAREER DEVELOPMENT CENTER 
PRODUCT EVALUATION 



Objectives of the Program : The Skyline Career Development Center is the 
product of the Dallas Independent School District's commitment to career 
education. The facilities and equipment are modern and representative 
of that found in the various careers for which the students are receiving 
training. (There are 28 clusters representing many different career 
areas. ) 

Curriculum continues to be developed for the Center, which may have impact 
throughout the District. The curriculum was to be unique to the District. 
It was also meant to be flexible, enabling students to perform at varying 
levels and terminate at different points in any of the offerings. 

As part of the concept of the Career Development Center as a catalyst 
for positive change, extensive efforts were made to provide criterion- 
referenced assessment instruments. The instruments were designed to 
measure success on the curriculum. 

Finally, the program was to foster more favorable attitudes toward school 
and encourage students to select and develop career skills. 

Purpose of the Evaluation ; Product v valuation (derived from the CIPP 
model) was designed to measure and interpret student attitudes and 
performance both at the conclusion of the project's second year of 
operation as well as during the school year. Also included in this 
portion of the evaluation was the examination of instructor and parei>t 
attitudes. This would allow decision-makers to carry out a change 
process with alternative^ options available depending on the findings 
gained through the evaluation. 

Sample : All or a portion (depending on the study) of the student body, 
parents ot the students, and the instructional staff at the Career 
Development Center during the 1972-73 school year were included in 
product evaluation. 

Evaluation Design ; The overall strategy was to obtain attitudinal and 
performance data on a wide variety of topics dealing with the Career 
Development Center and its operation. Personal interviews, survey 
questionnaire, student records, and a testing program provided the 
information for this part of the evaluation. Data were collected 
throughout the 1972-73 school year as required by each individual study. 
The five major evaluation questions under product evaluation 
as of July, 1973, were: 

1. What were the attitudes of students and parents toward 
the present form used for reporting student progress? 

2. What were the attitudes and/or perceptions of students 
toward the bussing to and from the Career Derelopment Center? 
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3. What was the student performance on the stated objectives 
for the cluster? 

4. What werp the students' attitudes toward, and perceptions of, 
class, career, and the Center itself? 

5. How did counselors and students perceive the counseling 
program? 

6. Whac: were the activities of the 1971-72 school-year graduates 
six months after graduation? 



Evaluation. Resul ts t 

1. Of the 323 criterion-referenced rest items evaluated, 144 
were deemed inappropriate for future testing. The major 
reason for suggesting inappropriateness was incorrect type 
of item (e.g., paper-pencil type instead of performance). 

2. Students in general, and minority students in particular, 
continued to view the Career Development Center in a most 
positive light. They voiced greatest approval of the 
individualized instruction taking place at the Center. 

3. The Student Progress Report Form, while favored over 
conventional reporting, was deemed in need of revision 
by students and parents in order to make it more 
communicable. 

4. Parents viewed the Career Development Center as being 
open. They felt it was generally easy for parents to 
interact with the staff at the Center. Blacks, more than 
any other ethnic group, expressed this sentiment. 

5. Students who relied on bussing for transportation were 
found to be generally satisfied with the existing bussing 
system. 

6. Almost two-thirds of all CDC students utilized the 
Counselor's Office, and virtually all of them received 
the help they were looking for. However, part-time 
students seldom use the counseling facilities at the 
Center. 

7. Only 3% of " the former students were found to be 
unemployed as of March, 1973. Of those employed, almost 
one-half were employed in areas related to their work 

at the Center. 
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Recommendations of Evaluators : 



1. Instruction should continue to be individualized when- 
ever and wherever feasible* 

2. Assistance should be given to students in finding a 
job suitable to their training. 

3. The individual student-progress reporting concept should 
be continued. However, efforts need to be made to 
assure easier understanding of the forms. 

4. Extensive effort should be made to provide an appropriate 
"criterion-referenced" assessment instrument for each 
behavioral objective. 

5. Alternative methods still need to be sought in 
communicating with minority parents. 

6. Additional studies should be undertaken to tannine 
possible retention problems involved in auditing student 
performance on stated objectives. 

7. Part-time students should be encouraged to take 
advantage of the counseling assistance available at 
Skyline. 

8. Further follow-up should be conducted on the graduates 
of 1971--72. 
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Additional Information : 

72-105 Plan for the Evaluation of the Skyline Career Development 

Center 1972-73, Thomas G. Krueck, 21 pp. 

S72-73-7 Asse?^ nent of Conditions and Limitations at -line CDC 
with reference to Criterion-Referenced Testing as of 
November 1, 1972 , Robbie G. Davis, 15 pp. 

S72-73-14 Parent Impressions of the Student Progress Reporting Form, 
1973, John Robert Clements, 13 pp. 

S72-73-16 Career Development Center Student Opinion Survey, 1973, 
Dan Peterson, 32 pp . 

S72-73-21 Testing Results of the Armed Services Aptitude Battery 
Administered at the Skyline CDC, 1973, Robbie G. Davis, 
54 pp. 
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S72-73-29 



Preliminary Graduate Student Report, 1973, Walter J, Hardi, 
14 pp. 



S72-73-33 Career Development Center Parental Response to "The Parent 

School Communications Questionnaire/* 1973, John R. Clements, 
Leonard Pruitt, 13 pp. 

S72-73-34 Final Report for Criterion-Referenced Testing at the Skyline 
Career Development Center for the Acadamic Year of 1972-73, 
Robbie G. Davis, 42 pp. 

S72-73-35 Student Perceptions and Impressions of the Career Develop- 
ment Center, 1973, Dan Peterson, 31 pp, 

S72-73-36 A Study of Retention Based Upon Results of Two Test Adminis- 
trations in Different Clusters at the Skyline CDC, 1973, 
Robbie G. Davis, 6 pp. 

Also interim Criterion-Referenced Testing Reports: 



S72-73-10 
S72-73-12 
S72-73-17 
S72-73-23 



S72-73-27 
S72-73-31 
872-73-32 
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FINAL AUDIT REPORT ON CURRICULUM DEVELOPMENT 
AT THE CAREER DEVELOPMENT CENTER 



This report discusses the audit of RCA Edu^ ition Systems curriculum 
by the Skyline-CDC Research and Evaluation unit» 

During the 1972-73 school year, the audit was carried out in 13 
clusters for which RCA had the responsibility of developing second- 
year curriculum. First-year curriculum was audited in two addi- 
tional areas. 

Until June, 1973, the Research and Evaluation Department also audited 
curriculum developed by the DISD Career Development Center Curriculum 
Department. Officially, four clusters were involved ^ but additional 
work .was done in several other areas as a service function. Audit of 
DISD-developed material was discontinued primarily because the "con- 
cept approach" to the curriculum development, as utilized by DISD-CDC 
curriculum writers, did not lend itself to audit, using the guidelines 
developed for auditing the RCA curriculum. 

Purpose of the audit function was to ensure measurability of behavioral 
objectives and the efficacy of their corresponding criterion-referenced 
assessment instruments. Other requirements of concern to Research and 
Evaluation included standardization, specificity, realistic time frames, 
and appropriate and acceptable methods of measuring or judging task 
completion. For a more detailed outline of guidelines utilized by Re- 
search and Evaluation, refer to the body of this report. 

To date, some 1,055 behavioral objectives and 740 criterion-referenced 
assessment instruments have been reviewed. Additional instruments are 
still being submitted. 

The major results of the audit function appeared to show that RCA was 
successfully fulfilling their contractural obligations. 



Additional Information : 

73-202 Final Audit Report on Curriculum Development at the 
Career Development Center, Thomas Krueck, 10 pp. 
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BASAL READING PROGRAM: 
KEYS TO READING 



Objectives of the Program : Keys to Reading Is a basal reading program which 
emphasizes the development of word-analysis skills through a contextual- 
phonetic approach. The fundamental objectives of the Program are to develop 
competence in basic aspects of the reading process, to engage in general 
learning functions appropriate to reading, and to apply reading skills and 
understandings. Thus, the overall Program is expected to have a significant 
impact on the development of both cognitive and affective reading skills. 

Purpose of the Evaluation : The evaluation of the Keys to Reading Program 
was designed to determine how effective the Program was in comparison to the 
primary DISD basal reading program, Houghton Mifflin. Evaluation results 
were expected to be used as input to decide if the Keys to Reading Program 
was sufficiently effective to warrant its continuation or expansion to 
other schools . 



Sam ple : Four elementary schools, two experimental sites and two comparison 
sites, were included in the study. Experimental schools were Nancy Moseley 
and Seagoville, with Richard Lagow and Julius Dorsey as comparison schools* 
The Keys to Reading Program was implemented in grades one through three'at 
the experimental schools, with comparison schools providing instruction 
under the Houghton Mifflin reading program. 



Evaluation Design : A nonrandomized control-group pretes t^pos t tes t de. i^^n 

Pretest Treatment Posttest 



Experimental 
Comparison 



Tj X T2 

Ti ' T2 



was used in assessing students' cognitive reading skills. Since comparison 
and experimental scudent populations were carefully matched according to 
ethnic composition, socioeconomic level, and past reading achievement, the 
internal validity of the design was considered satisfactory. In addition, 
further control of the data was obtained using "blocking** or '^leveling" 
procedures. 

A control-group pos ttes t-only design was used to analyze individualization 

Pretest Treatment Posttest 



Experimental 
Comparison 



X T2 



of instruction and affective reading data. Pretest data v/ere not available 
due to late implementation of the evaluation design. However, the validity 
of the design would seem to be adequate due to the careful matching of 
experimental and comparison student populations. 
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The major evaluation questions addressed were: 



1. What was the organization of the Keys to Reading Program 
in those schools where it was piloted? 

2. Was there greater individual! 7 "tion of instruction under 

Keys to Reading than with the lioughton Mifflin reading program? 

3. Did students in the Keys to Reading Program show a greater 
level of performance on a standardized measure of reading 
achievement than students in the Houghton Mifflin reading 
program? 

Evaluation Results ; 

1. There was the highest degree of implementation of the 
Keys to Reading Program. 

2. In comparison to Houghton Mifflin teachers. Keys to Reading 
teachers perceived their reading Program as being more 
effective in developing decoding/word perception skills 
and study/reference skills. 

3. There was no apparent difference in the degree of 
individualization of instruction between the Keys to 
Reading and Houghton Mifflin programs. 

4. With the exception of "high" ability students at the 
second-grade level, Keys to Reading students exhibited 
significantly greater achievement than Houghton Mifflin 
students. Achievement differences were most striking 
at the first-grade level, with Keys to Reading students 
exhibiting an especially large achievement advantage. 

5. Affective reading measures were found to be invalid, 
and consequently no attempt was made to interpret the 
data. 



Recommendations of Evaluator : 

1. The Keys to Reading Program should be continued at 
Nancy Hoseley and Seagoville. 

2. The evaluation effort should be continued in order 
to assess long-term effects of the Program. 

3. The Program should be extended to other pilot schools 
having disadvantaged student populations, with 

careful evaluation of its effectiveness in those settings. 
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4. If the Program is expanded, careful planning should be 
undertaken to ensure a Program design which will readily 
lend itself to evaluation. 



5. Instruments to assess students' affective reading skills 
accurately should be identified or designed. 

Addi tional Inf ofmation ; 

72- 112 Evaluation Design for Basal Reading Program: Keys to Reading, 19 

C. LaVor Lym, 48 pp. 

73- 144 Basal Reading Program: Keys to Reading, 1973, C. LaVor Lym, 77 pp 
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RIGHT TO READ PROGRAMS 



Objectives of the Program : The school-based National Right to Read 
Effort (NRRE) was aimed at having selected schools assess the reading 
needs of their pupils, critically review their present reading program, 
and reorganize their program to fulfill the defined needs. Although 
materials and equipment could be obtained with Right to Read (R2R) funds, 
the NRRE did not involve the adoption of a "specified" curriculum. 
However, implementation of a "diagnostic-prescriptive" approach to 
instruction was espoused. 

In general, R2R students were expected to show improved performance on 
cognitive reading tasks and to demonstrate increased motivation toward 
reading or school, R2R teachers were expected to gain proficiency in the 
application of diagnostic-prescriptive techniques, and parents of R2R 
children were expected to become knowledgeable about the program and 
actively participate in program planning and implementation. 

Purpose of the Evaluation : Evaluation of the R2R programs within the 
DISD was designed to define the environments of the programs, assess the 
implementation of the programs, and measure student outcomes in relation 
to cognitive and affective reading skills and attitude-toward-school . 
Evaluation results were expected to provide program management with 
information which could be used to improve the effectiveness of the 
programs. 

Sample : Seven elementary schools, three R2R sites and four comparison 
sites, were involved in the evaluation. R2R schools were David Crockett, 
Paul L. Dunbar, and T. D. Marshall, while James Bowie, James W. Fannin, 
W. W. Bushman, and Erasmo Seguin were comparison schools. The R2R 
program at David Crockett was restricted to grades one through six, 
while program implementation was focused on grades four through six at 
Paul L. Dunbar. Grades one through three, or the total student popula- 
tion, at T. D. Marshall were involved in the program. 

Evaluation Design : A nonrandomized control-group pretest-posttest 
design 

Pretest^ Treatment Posttest 

Experimental 0^ X O2 

Comparison 0^ O2 

was used in assessing students' cognitive reading skills. In the 
configuration for this design, X represents the implementation of R2R, 
while 0^^ and O2 represent, respectively, pretest and posttest observa- 
tions. Since comparison and experimental students were matched 
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according to ethnic composition, socioeconomic level, and past reading 
achievement, the internal validity of the design was considered satis- 
factory. In addition, further control of the data was obtained using 
'^blocking" procedures. 

A pretest-posttest design 

Pretest Treatment Posttest 
Experimental 0^^ X O2 

was used to examine "within group" changes in achievement by R2R 
students . 

Data related to individualization of instruction, teachers' ratings of 
their reading programs, attitude-toward-reading, and attitude-toward- 
school were obtained using a control-group posttest-only design. 

Pretest Treatment Posttest 
Experimental X O2 

' Comparison 

The major evaluation questions of interest were: 

1. To what extent was the "diagnostic-prescriptive" approach 
applied by the R2R teachers? 

2. What was the extent of parental awareness of the DISD R2R 
programs? 

3. Were R2R classrooms characterized by a higher degree of 
individualization of instruction than comparison classrooms? 

4. Did teachers' ratings of the R2R program differ from comparison 
teachers ' ratings of their reading program on dimensions of 

a. decoding or word-perception skills 

b. study/reference skills? 

c. critical thinking (evaluation)? 

d. comprehension? 

e. reading aesthetics? 

5. Did R2R students show a change in performance from pretest to 
posttest on 

a. a standardized test for reading achievement? 

b. a selected measure of attitude-toward-reading? 

c. a selected measure of attitude-toward-school? 

6. Did R2R students perform at a higher level than non-R2R 
students on 
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teachers in applying uniform diagnostic techniques and formu- 
lating instructional prescriptions. Charts or folders reflect- 
ing placement (as based on diagnosis), prescribed instruction, 
and attainnient or mastery of the prescribed work should be 
maintained for individual students. 

2. The evaluation of the R2R programs should be extended, with a 
greater emphasis on process evaluation, assessment of community 
involvement, and the examination of affective reading constructs 

3. To improve the quality of reading achievement data, all R2R 
teachers should be exposed to a special inservice workshop 
designed to reinforce the knowledge and application of skills 
in test administration. 

4. The application of criterion-referenced tests in assessing the 
R2R programs should be considered. In many R2R classrooms, 
curriculum materials are being used for which criterion-refer- 
enced tests already exist. For other classrooms, criterion- 
referenced tests may be adopted or designed. 

5. Any new teacher making the transition into one of the R2R 
programs should receive special assistance and inservice 
training. 

6. Although the data clearly indicate an especially low achieve- 
ment gain for first-grade students at David Crockett, the 
source of this finding remains unknown. The uniformly low 
level of performance demonstrated by Anglo, Negro, and Mexican- 
American students would suggest either a breakdown in the test- 
ing procedure or serious deficiency in the instructional pjic- 
cess. A careful investigation should be made to determine 
that source which actually gave rise to the finding. 

7. A basic-concepts test written in Spanish should be used in 
testing Spanish-dominant Mexican-American students at David 
Crockett. Testing with this instrument should replace testing 
with standardized achievement tests . 

8. The factors contributing to the extremely low level of perform- 
ance demonstrated by fifth-grade students at Paul L. Dunbar 
should be identified and necessary steps taken to remedy the 
problem. 

9. InstruToents to assess students' affective reading skills 
accurately should be identified or designed. Any instrument 
adopted should exhibit sufficiently high reliability and 
validity to give confidence in the results obtained from its 
use. 
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a. a standardized test for reading achievement? 

b. a selected measure of attitude-toward-reading? 

c. a selected measure of attitude-toward-school? 



Evaluation Results ; 

1. Although R2R teachers showed extensive implementation of 
curriculum materials and equipment, there was little imple- 
mentation of the diagnostic-prescriptive instructional 
approach. 

2. The estimated percentage of parents who were aware of the R2R 
program at David Crockett and Paul L • Dunbar was approximately 
^0%. A somewhat higher percentage of parents at T. D. Marshall, 
67%, was estimated to be aware of the R2R program at the 
school • 

3. There was no apparent difference in the degree of Individuali- 
zation of instruction between R2R and comparison classrooms. 

4. In comparison to non-R2R teachers, R2R teachers viewed their 
program as being significantly more effective in developing 
study /reference skills, critical thinking, comprehension, and 
reading aesthetics. Especially large differences in ratings 
were noted for reading aesthetics and critical-thinking skills. 

5. Achievement gains were generally below the expected level 
(i.e., a month gain in grade equivalent for each month of 
instruction) except for second-grade students at David Crockett. 

6. In general, Mexican-American students in the R2R program 
perfox'raed at a lower level on the standardized reading tests 
but made gains comparable to "other'* students, 

{• 

7. As compared to similar students not involved in the program, 
R2R students, in general, showed no better performance on 
standardized reading tests and, in some instances, exhibited 
significantly lower achievement. 

8. Assuming validity of school-based attitude measures, no 
significant difference was noted between R2R and non~R2R 
students for this criterion. 



Recommendations of the Evaluator ; 

1. A greater implementation of the diagnostic-prescriptive 
technique within the classroom is recommended. Technical 
assistance should be offered at the building level to assist 
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Additional Information ; 

72- 111 Evaluation Design for Right to Read Programs, 1972-73, 

C. LaVor Lym, 54 pp. 

73- 140 Evaluation Report: Right to Read Programs, 1972-73, 

C. LaVor Lym, 107 pp. 
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SUPPORTIVE SERVICES 
FOR INDIVIDUALIZED INSTRUCTION 
SPECIAL EDUCATION 



Ob:)ective of the Program : The purpose of Supportive Services is to provide 
exceptional children with an individualized educational plan which uses one 
or more instructional arrangements to integrate the exceptional child, when- 
ever feasible, into the regular curriculum* The use of local Admission, 
Review, and Dismissal (ARD) Committees, composed of local school personnel 
and itinerant appraisal personnel, to deliver special education services 
was a radical revision of the traditional special education model. It is 
hoped that Supportive Services can extend special education services to 
all eligible students and can contribute to providing these students with 
a successful school experience. 



Purpose of the Evaluation : The evaluation concentrated heavily on process 
evaluation to determine the success of program implementation, since this 
was the first year of operation. The evaluation attempted to monitor the 
implementation process and to provide feedback to program management. Pro- 
duct evaluation considered student improvement in academic achievement 
and attendanc^. 



Sample : The program was implemented in the Thomas Jefferson High School 
cluster. The eleven schools included were: T. Jefferson High SchooL, Cary 
Junior High School, Walker Middle School, Caillet, Foster, Burnet, Field, 
Longfellow, Walnut Hills Polk, and Williams Elementary Schools, 

Evaluation Design ; The evaluation relied on a system analysis approach to 
monitor program operation and implementation. Operational records from ARD 
committee meetings provided information for process evaluation. Product 
evaluation used a pre-post measurement design in which the student's previous 
performance constituted the control. The major evaluation questions wera: 

1. What characteristics described the implementation and operation 
of the ARD Committees? 

2. What V7ere the attitudes and opinions expressed by faculty 
members toward "Supportive Services in schools scheduled for 
1973-74 implementation? 

3. What was the effect of receiving Supportive Services with 
respect to academic achievement and attendance? 

4. Wliat was the reaction of ARD personnel to the Supportive 
Services program? 

5. What were the awareness and reaction of parents whose children 
received Supportive Services? 
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Evaluation ResuJ.ts : 



1. The program encountered major implementation difficulties. 
The predominant implementation problems were the absence 
of clearly defined operational policies and procedures, 

a tremendo\is backlog in the development of educational plans, 
and delays in the procurement of instructional materials. 
Management initiated a revised implementation model in 
February, 1973, which greatly improved program implementa- 
tion. There was considerable variation in the succes s of 
program implementation among schools. 

2. Supportive Services served fewer students than expected, and 
there was no conclusive evidence that the student popula- 
tion served was necessarily appropriate as defined by program 
specifications. The typical student in the program was 
Anglo male and had a history of poor academic performance 
and poor attendance. The great majority of students had 

not previously received special education services. 

3. Shortcomings in program implementation hampered delivery of 
services to students. Most students received instructional 
assistance in language arts, math, and perceptional skills 
through part-time attendance in a resource room. Counseling 
^n>5 bu<ot ''isits were also utilized, but numerous services 
del^veref' by Supportive Services personnel were not docu- 
mented in the ARD records. The majority of students were 
retained in the program for September, 1973. 

4. Educational and psychological testing comprised most of the 
information-collecting activities of ARD committees. There 
was great variability among schools in the student appraisal 
process. 

5. The Supportive Services personnel, parents, and regular 
faculty in schools scheduled for new implementation responded 
favorably to the program. 

6. Subjective reports from program staff indicated that most stu- 
dents had shown improvement on one or more variables. There 
was objective evidence of important reading gains in one 
elementary school, and there was a trend toward improved 
attendance in a sample of all elementary students. 

Recommendation of Evaluator : 

1. Develop more specific program objectives. 

2. Develop and communicate clearly defined operational policies 
and procedures. 
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3- Develop and implement a record-keeping system v/hich will 
be appropriate for all Supportive Services personnel and 
which will accurately provide all information needed by 
management for program operation and accounting. 

4. Develop and in^pl^ment a more precise means of deter- 
mining student eligibility to ensure that the appropriate 
population receives Supportive Services. 

5. Provide Supportive Services personnel and related personnel 
(regular classroom teachers) with preparation and orientation 
to ensure understanding oif professional functions and 
responsibilities. 

6. Provide resource teachers with an adequate instructional 
materials inventory at the beginning of the school year. 

7. Ensure that all Supportive Services students receive edu- 
cational plans which specify instructional objectives, 
methodology, materials, and means for observing attain- 
ment of objectives- 

8. Consider differential allocation of resources to meet 
needs of varying student subpopulations. 



Additional Information : 

72- 113 Evaluation Design for Supportive Services - Special 

Education, 1973, Daniel J. Macy, 38 pp. 

73- 135 Evaluation Report for Supportive Services for 

Individualized Instruction Special Education, 1972, ' 
Daniel J. Macy, 61 pp. 
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EVALUATION OF DRUG EDUCATION PROGRAM 



Ob.jectlves of the Program ; The major objectives of the Drug Abuse 
Education Program (DAEP) were to inform students of the facts about 
drugs; to make students aware of the personal, social, and economic 
problems leading to the misuse of drugs; to inform students of the 
consequences of a drug-centered existence; and to promote character 
and moral values which will be ultimate deterrents to drug abuse. 



Purpose of the Evaluation : The major purposes of the evaluation of 
the DAEP were to assess the instructional impact of the Program, to 
provide information concerning the implementation of the Program, 
and to examine various pilot projects within the Program. 

Sample : Nine elementary schools in Area I (far North Dallas) were 
stratified according to socioeconomic level of student populations 
and randomly assigned to one of three instructional modes: DAEP, 
Instructional Implementation, and Drug Division. Seventh-grade 
health- science teachers at these schools implemented the assigned 
program. 



Evaluation Design : The pretest-post test design 

Pretest Treatment Posttest 



DAEP 

Instructional Implementation 
Comparison 



o; 



was used to compare the effectiveness of the types of instruction, with 
knowledge of drug facts, attitude toward drugs, and sentiment regarding 
drug abuse education as the criteria. 

The three staff development seminars (cognitive skills, valuing, and 
eclectic) for the DAEP were compared, using a pretes t-posttest design 
similar to the one given above. 

Implementation of the DAEP was investigated by making a census of 
Instructional Implementation teachers and through a survey of a randomly 
selected sample from the population of all Dallas Independent School 
District (DISD) teachers. Youth-to-youth, a pilot project implemented 
by W. T. White High School students, and the initial phase of the 
Valuing program at E. D. Walker were examined .by sampling the opinions 
and respective student and teacher target groups. 

The major questions addressed in the evaluation were: 



ERLC 



90 



1. What were the differences in perfotTcnance among DISD students 
who received instruction under the DAEP, the Area I Instruc- 
tional Implementation program, and Drug Decision for the 
following criteria: 

a. drug knowledge? 

b. attitude toward drugs? 

c. opinions concerning drug abuse education? 

2. To what extent was the DAEP implemented District-^idc? 

3. To what extent was instruction in drug abuse education imple- 
mented by designated teachers in the Instructional Implementa- 
tion program? 

4. Were there differences among the three approaches to staff 
development in relation to acceptance of the philosophy of 
drug abuse education by participants? 

5. Were nhere differences among the three approaches to staff 
development in relation to the acquisition of drug facts 
by participants? 

6. Did participants perceive a difference in the effectiveness 
of the three approaches to staff development? 

7. What were teachers' attitudes regarding training for imple- 
menting the Valuing program? 

8. W\at were the opinions of the target student group concerning 
the Youth-to-Youth program? 



Evaluation Results ; 

1. All seventh-grade students who received instruction under the 
Drug Abuse Education Program (DAEP) , the Area I Instructional 
Implementation program, and Drug Decision showed improved 
performance from pre- to posttest on a test of drug facts. 
This improvement, however, was greatest for the Instructional 
Implementation program, followed by Drug Decision, and then 
the DAEP. 

2. There was no change in performance on a selected measure of 
attitude toward drugs from pre- to posttest for any of the 
three instructional modes. Pretest means, which reflected 
positive attitudes toward drugs prior to instruction, did not 
differ among the three groups. 
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3. Drug Decision students viewed drug abuse education as more of 
a "waste of time" and were less interested in extending their 
learning about drugs than DAEP or Instructional Implementation 
students . 

4. DAEP and Drug Decision students believed more strongly than 
Instructional Implementation students that '^the most important 
reason for teaching about drug abuse is to find those students 
who use druga . " . 

5. The group of teachers in Area I who were designated to work 
with the DAEP management in the Instructional Implementation 
program reported extensive implementation of drug abuse educa- 
tional activities. 

6. In assessing District-wide implementation of the DAEP for 
1972-73, an estimated 50% of all elementary teachers 

did not provide any instruction in drug abuse education. 
Approximately 20% of those sampled in the system-wide 
survey reported spending three or more hours of instruction 
in drug abuse education. 

1 . Implementation of the DAEP by secondary teachers varied 

widely according to subject area. Estimated implementation 
was highest for science teachers, with about 60% of 
those sampled reporting three or more hours of classroom 
instruction. Implementation (three or more hours) for social 
science teachers was 35% , compared to approximately 10^ for 
language and mathematics teachers. 

8. The eclectic approach taken in the Area I staff development 
seminar appeared to be much more effective than the cognitive 
skills approach in Areas II and IV or the valuing approach 

in Area III. 

9. Although teachers perceived the Valuing program as worthwhile, 
they expressed negative opinions concerning the organization 
of the sessions to provide training for implementation of the 
program. 

10. Target students in the Youth-to-Youth program expressed 

positive opinions regarding the peer influence approach to 
instruction. They were especially optimistic about the 
effects of the program in combating drug experimentation. 

Recommendations of the Evaluator : 

1. Further implementation of the Drug Decision program is recom- 
mended since materials for the program are available. However, 
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to overcome the weaknesses in the program evidenced hy teachers 
and students, it is recommended that: (a) the length of the 
program be reduced, (b) appropriate substitute films be selec- 
ted for the "cartoon" films, and (c) the implementation be 
carefully coordinated to ensure the dispensing of materials 
to meet the teachers' needs. 

2. The Drug Decision program should not be implemented in class- 
rooms where students are known to have limited reading skills, 

3. In extending implementation of the DAEP, special emphasis 
should be focused on teachers in self-contained elementary 
classrooms. Ifliere there is subject-area specialization (upper 
elementary and secondary), social science or science teachers 
should be expected to show major implementation of the program. 
Therefore, special communication should be made with these 
teachers regarding program implementation, 

4. The eclectic approach, which emphasizes instructional skills, 
cognitive skills, and valuing, should be applied in any future 
staff development activities for teachers. 

5. The valuing program at E. D. Walker should be carefully 
reviewed before moving into the Instructional Implementation 
phase. If necessary, additional staff development sessions 
should be held or, possibly, alternative sites for implement- 
ing the program should be considered. 

6. The Youth-to-Youth program should be continued, and, whenever 
possible J integrated into the teachers' instructional unit on 
drug abuse. 



Additional Inf ormation : 

72- 93 Evaluation Design for the Drug Education Program, 19 72-73, 

C. LaVor Lym, 23 pp. 

73- 155 Evaluation of Drug Education Program, 1972-73, C. LaVor Lym, 

75 pp. 
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STUDY OF ALTERNATIVE GRADING 
AJID REPORTING PRACTICES 



Objectives of the Program ; The Dallas Independent School District has 
several current efforts to develop new methods of reporting student 
progress. These efforts include: grades one through three, the Bilin- 
gual-Multicultural Education Program and the Early Childhood Education 
program; grades five through eight, the Continuous Progress system at 
E. D. Walker; and grades ten through twelve, the Continuous Progress 
system at Pinkston and the Behavioral Objective system at the Skyline 
Career Development Center. 



Purpose of the Evaluation ; This study was designed to ascertain stu-- 
dent, teacher, and parent opinions of the various forms being used in 
the District to report student progress. 



Sample ; Because of the extensive number of schools involved in some of 
the innovative programs, a sample of schools was selected on a geographic 
basis. Within the sample of schools, all teachers were surveyed and a 
sample of classrooms was selected for student and parent representation. 

Evaluation Design ; The above sample of students, teachers, and parents 
was surveyed to determine their opinion on the reporting forms used. 
Different techniques were used to obtain responses from parents, vary- 
ing from interviews to having children take questionnaires home. The 
major evaluation questions of interest were; 

1. Does the reporting form convey an accurate description 
of student progress? 

2. Can parents understand the forms? 

3. What recommendations would parents make for improving 
the reporting system? 

Evaluation Results ; 

1, Teachers in most programs do not feel the present reporting 
form provides an accurate descript:ion of how well a child 
is doing. The notable exceptions are the systems used with 
the Bilingual-Multicultural Education Program and the 
Career Development Center. 

2. Parents, in all cases, feel the forms do present an accurate 
description of how well the child is doing. 



ERIC 



3. In all innovative systems, teachers say that parents do not 
understand the reporting form being used. 

4. In all systems, parents say the forms are easy to understand. 

5. Parents would like to see more teacher comments on the 
reporting forms and have parent-teacher conferences. 

Recommendations of Evaluator : 

1. Efforts to develop innovative, individualized student progress 
reporting forms should be continued. 

2. Before individualized student reporting can become more 
widespread , individualized criterion-referenced curriculum 
needs to be established. 

3. Efforts should be increased to find viable ways to incorporate 
parent-teacher conferences. 

4. Whatever innovative student progress reporting systems are 
developed, they must not increase the present workload of 
teachers. 

5. Extensive parental involvement needs to be incorporated into 
future plans to develop student progress reporting systems. 



Additional Information ; 

73-145 Study of Alternative Grading and Reporting Practices, 1973, 
William T. Denton, 67 pp. 
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EMERGENCY SCHOOL ASSISTANCE PROGRAM 



Objectives of the Program ; The Federal Court order of August 9, 1971, 
relative to desegregation in the Dallas Independent School District 
(DISD) , generated new needs which required redirection and expansion 
of existing programs and the creation of new programs. To facilitate 
this demand, the 1971-72 Emergency School Assistance Program (ESAP) 
was funded through the United States Office of Education and consisted 
of these seven components: (1) Staff Development, (2) Student Affairs, 
(3) Youth Advisors, (4) Multicultural Education, (5) Principal's 
Aide, (6) Community Affairs, and (7) Multiethnic Music. Each compon- 
ent had specific goals which comprehensively projected the attainment 
of a true cultural confluence among school personnel, pupils, parents, 
and the community. 

Evaluation Design : Considering that the 1971-72 school year was 
basically a year of implementation of the ESAP, evaluation was largely 
one of process approach. The sample consisted of all persons partici- 
pating in the ESAP. Each of the seven components was monitored, and 
information was compiled relative to the goal operations and evalua- 
tion questions. Survey-type instrun.ants wf^re utilized to solicit 
responses from key personnel and targeted students in each component. 

Evaluation Results : Results are stated relative to each of the seven 
ESAP components. 

Staff Development. The following Dallas Teacher Education 
Centers were established and in partial operations as of May, 1972 r 
H, S. Thompson, Longfellow, Silberstein, Paul L. Dunbar Learning Lab, 
Birdie Alexander, and North Dallas. These centers functioned in 
training, resource, and cultural capacities. Three groups of class- 
room teachers, and one group of supervisory teachers and student 
teachers, indicated somewhat favorable responses toward the H. S. 
Thompson program* A number of the Thompson participants, however, 
related adverse feelings toward the selection process for entry into 
the program. 

A Confluence of Cultures staff development program was held in 
August, 1971, at Skyline Center and an attitude survey designed by 
Research and Evaluation (R & E) was administered at that time. The 
survey indicated generally positive feelings toward desegregation 
and the District's administrative staff. The survey was not admin- 
istered in the spring. 

Student Affairs. The Superintendent's Student Advisory Committee 
(SUPER-SAC) met with the General Superintendent approximately once 
during every six-to-eight week period of the school year. The 
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committee was composed of two student representatives from each high 
school. A questionnaire concerning the effectiveness of SUPER--SAC 
was administered to 30 student members of the committee on March 16, 
1972. Responses toward the committee were generally favorable. There 
were strong indications that the committee meetings were more bene- 
ficial to the committee members, themselves, than to the students 
that the members represented. A high number of students were unde- 
cided about meeting student needs as a result of the committee's 
exchange of ideas. 

In order to facilitate better student-teacher relations through 
improving teacher self -concept, the ZZOOOMMM Program was implemented 
in ten schools of the District. Eleven teachers at each school par- 
ticipated in the sessions which began either during December, January, 
or February of the 1971-72 school year. Evaluation measures were 
planuad at the Program's initiation. These measures consisted of 
taped interviews, the Jim Scale (attitude measure), and a question- 
naire. For the most part, specific information from the measures 
was held confidential. 

Youth Advisors. Thirty-four youth advisors were assigned to 28 
secondary schools for the purpose of designing adjustment programs 
for students. An opinion survey was designed by Research and Evalu- 
ation and administered in May, 1972, to a stratified, random sample 
of students who were expected to be served by youth advisors. Twenty- 
seven schools administered the opinion surveys and returned 73% of 
the 2,173 surveys distributed. Students generally indicated that one 
or more of these services was rendered: (a) private counseling, 
(b) consulting with parents, (c) adjusting class schedule, (d) counsel- 
ing concerning relationships with fellow students, and (e) adjusting 
to riding the bus to school. 

Responses were generally favorable toward the youth advisor's 
services. The student responses, however, did cluster according to 
school; students at certain schools were distinctly more favorable 
than students at other schools. There were approximately equal posi- 
tive and negative reactions to the question of needing the youth 
advisor next school year. At least two-thirds of the 27 schools had 
returns in which there were more students reporting some contact 
with the youth advisor than there were students reporting no such 
contact. Two youth advisors were reassigned for the 1972-73 school 
year. 

The tutoring program was operational in 23 secondary schools, 
which included three night schools. Evaluation of the tutoring pro- 
gram was the primar}'' responsibility of the program coordinator, 
Mr- Jim Daniel. According to Mr. Daniel's evaluation, little objec- 
tive data was available to support the general concensus that the 
program was successful. Questionnaires regarding the effectiveness 
of the program were administered to the students tutored, tutors, and 
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school administrative personnel. These questionnaires solicited 
highly favorable responses. Specific data from some schools were 
compiled. For example, results from Sequoyah Middle School indi- 
cated a gain in reading ability by most students where previous 
records indicated a loss in reading ability. There was some evi- 
dence that attendance improved among the tutored Sequoyah students. 
Information from the program, as a whole, has suggested that student 
attitudes were improved. 

Multicultural Education . Resource materials were recommended 
and purchased for the six Teacher Education Centers. The materials 
were mainly a teacher-training resource which included some pupil 
materials. Three curriculum writers who will develop multicultural 
social education curriculum are funded under ESAP. 

Principalis Aide. Thirty-six teacher aides were assigned to 15 
elementary and 13 secondary schools. Evaluation forms concerning 
the services of the 36 teacher aides were sent to principals of 28 
schools on May 28, 1972. Thirty-three of the forms were completed 
and returned. Most of the forms indicated that teacher aides assisted 
teachers in individualizing instruction and in rendering other student 
services. The aides usually began work either in January or February, 
1972. Over half of the respondents indicated that the aides exhibited 
positive attitudes and cooperated well with school personnel. A few 
forms stated that some assigned aides never arrived to begin work and 
that others had left the job following only a short period of work. 

Communit y Affairs. The first administration of the public aware- 
ness-attitude" survey , which deals specifically with desegregation, 
was made during the latter part of the first semester (1971-72) . The 
District contracted with Louis, Bowles, and Grace, Inc., to do the 
survey. The second administration of the same survey was conducted 
during the suminer, 1972. A report of the results of the surveys was 
released by the Board of Education in late fall, 1972. 

A community-relations professional, Mr. Robert Watkins, assumed 
his duties on January 24, 19 72. His duties have included the organi- 
zation and coordination of: (1) after-school recreation programs, 
(2) student tours, (3) school exchange programs involving parents, 
(4) booth exhibits in various school communities, and (5) Ethnic 
Arts Festival. 

Two of the proposed non-professional field workers, community 
liaison persons, were employed part-time on March 15, 1972 (approxi- 
mate date). These two persons reported to Mr. Watkins and basically 
served as communicators between the school and the community. 

The District's first Ethnic Arts Festival was held on May 18, 
1972, at the Dallas Memorial Auditorium. The festival featured 
1,500 students of all ages and ethnic groups in a sampling of ethnic 
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songs, instrumental music, and dances. Outstanding guest performers 
representing multiethnic backgrounds also were Included in the 
festival. Narrator and producer was Tom Hughes, managing director 
and producer of the Dallas Summer Musicals. Student art, which was 
displayed throughout the auditorium, portrayed the theme of the even- 
ing. The festival attracted a diverse and capacity audience. 

Multiethnic Music. Instruction in Mariachi (Mexican music) was 
provided for students at North Dallas High School, Spence Junior High 
Schoolj and Rusk Junior High School. These instructions began 
February 21, 1972 (approximate date)} and served approximately 56 
students. Tuition-free classes in Mariachi were piloted in sumirier, 
19 72 , at the secondary level. There was an insufficient number of 
students volunteering for Mariachi at Skyline High School, therefore 
the proposed class was not established. 

Instructions in Suzuki Violin were extended from the Paul Dunbar 
Elementary School to William Travis Elementary School on October 1, 
1971. The program was also begun at Edward Titche Elementary School 
on October 22, 1971. This extension provided instruction for an 
additional 95 pupils. ESAP funds were used to purchase instruments 
for the Travis and Titche programs. Five public performances in 
violin have been reported with student participants from the Suzuki 
program. These performances were all rated either **good" or "excel- 
lent" by community and school music personnel. Students in the 
Suzuki program also participated in the "massed strings" at the DISD 
Ethnic Arts Festival on May 18, 1972. 

Recommendations of Evaluator : Inconclusive evaluation results of all 
activities of the various ESAP components revealed a need for the 
constant monitoring oi" these activities. This procedure would further 
identify the planned and the implemented activities relative to 
accomplishing the proposed goals of each component. This process 
approach to evaluation would enable Program changes which would 
ensure more goal-directed activities for terminal examination. End 
results would facilitate decision-making processes and thus enable 
administrative personnel better to serve DISD students who wex£ 
specifically affected by . the desegregation order of August, 1971. 
Because a number of activities included in this evaluation have only 
been operational for a short period of time, a continual monitoring 
of ESAP over an extended period of time would also permit more valid 
and reliable results. 

The following ESAP components reflect the indicated recommenda- 
tions : 

1. Staff Development . Since teachers responded adversely 
toward the means of selection for participation in the 
Teacher Education Centers, it is recommended that the 
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present process for selection be examined for revisions. 



2. Student Affairs , Although evaluation revealed that there 
was significant student representation at SUPER-SAC meet- 
ings, members did indicate a need to capture input from 
more students besides themselves. Therefore, it is recom- 
mended that efforts be made to include more ideas from 
students in SUPER-SAC activities. 

3. Youth Advisors , The high number of students who did not 
know about the youth advisor suggests that a definite 
effort should be made to inform the targeted students of 
services available to them. 



Additional Information ; 

72-75 1971-72 Emergency School Assistance Program, 1972, Cordelia 
R. Alexander, 87 pp. 
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INDIVIDUALLY GUIDED EDUCATION 



Objectives of the Program ; Individually Guided Education (IGE) , developed 
by the Institute for the Development of Education Activities (/l/D/E/A/), 
began implementation in Dallas in February, 1973, in seven elementary 
schools. IGE is an approach to schooling that provides a framework for 
individualized instruction through a staff development program designed 
to reorganize and redirect the time, talents, and energies of an entire 
school. It has as one of its main purposes that of providing teachers 
with the mechanisms for developing appropriate learning programs for 
each child. 



Purpose of the Evaluation ; The evaluation of IGE was designed to deter- 
mine if its implementation promotes team-teaching and changes the mode 
of instruction toward the District's priority goal of individualized 
instruction. In addition, the evaluation design sought information on 
the suitability of IGK for implementation in other District elementary 
schools . 

Sample ; All teachers in the seven schools implementing IGE were included 
in the data-collection process. These schools were Birdie /Jexand^r, 
William M. Anderson, F. P. Caillet, T. L. Marsalis, John J. Pershing, 
Erasmo Seguin, and Ascher Silberstein. 

Evaluation Design : A questionnaire related to the evaluation questions 
was administered to all teachers prior to the beginning of IGE implemen- 
tation (February, 1973) and at the end of the school year (May, 1973). 
Comparisons were made between the two administrations. In addition, 
three schools were in the initial phases in May, 1973, and four schools 
evidenced higher levels of implementation. These two levels were com- 
pared to determine if the degree of implementation affected attainment 
of project goals. The major evaluation questions were: 

1. Does implementing IGE increase the level of team- 
teaching? 

2. Does implementing IGE make more time available for 
teacher-planning? 

3. Does implementing IGE increase the incidence of stu- 
dent assessment by teachers prior to and after instruc- 
tion? 

4. Does implementing IGE change the mode of instruction? 

5. Do teachers feel that chere exist opportunities for 
professional growth within the school? 



ERIC 



Evaluation Results: 



1. The implementation of IGE increased the level of team- 
teaching even in its initial stages and more so when 
moderate levels of IGE implementation were achieved, 

2. Initially, the time available for teacher-planning de*- 
creased but showed a slight gain after the initial 
phase was completed. 

3. There was a slight increase in teacher assessment of 
students prior to instruction and no change in post- 
instruction assessment. 

4. A slight increase in multiage grouping was reported: 
there was no change in the use of large-group, small- 
group, and one-to-one instructional techniques; and 

a marked increase in the use of independent study as 
an instructional mode was reported. 

5. Teachers felt that there are opportunities for pro- 
fessional growth within the school through formal 
inservice programs and through involvement in higher 
levels of professional decision-making. 



Recommendations of Evaluator ; 

1. Continue the implementation of IGE in Dallas. 

2. Provide encouragement and support to those schools 
in the initial implementation phases. Additional 
teacher-aides would help, if available. 

3. Provide specific inservice programs related to 
alternate modes of instruction. 

4. Insure that student independent study is not 
being used as a means of providing teachers with 
more planning time during the school day unless 
it is an effective instructional strategy. 



Additional Information : 

73-127 Evaluation Design for Individually Guided Education, 1972-73, 
Gary W. Womble, 10 pp. 

73-154 Evaluation Report: Individually Guided Education, 1972-73, 
Gary W. Womble, 26 pp. 
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OPEN-AREA SCHOOLS 



Objectives of the Program : Open-area schools were introduced in the Dallas 
Independent School District (DISD) in an attempt to increase individualized 
instruction and meet a growing demand for new facilities. Open-area school 
construction was initiated to facilitate new instructional and administrative 
philosophies and reduce building costs. 

Purpose of Evaluation ; The evaluation was designed to determine if open- 
area schools affected relevant student variables, i.e., did they help or 
deter student progress? 

Sample ; Approximately 8% of the students in all open-area schools and 
matched control schools were sampled. All teachers in open-area schools 
and the matched control schools were included in the sample. The control 
schools were matched on ethnicity, socioeconomic status, and instructional 
program. The open-area schools were Arlington Park, Darrell, Navarro, 
Tyler, Young, Jackson, Marshall, Seguin, and Buckner. The control schools 
were Carr, Carver, Earhart, Lisbon, Budd , Miller, Thornton, and Blair. 



Evaluation Design : The major evaluation questions were intended to provide 
data relative to the comparability of the schools used in this evaluation, 
teacher characteristics, student achievement and attitudes toward school, 
and organizational climate of the schools involved. Appropriate statistical 
techniques were used to measure any differences observed. 

Evaluation Results ; 

1. The schools involved in this evaluation were not representative 
of the District as a whole. 

2. The characteristics of teachers in open-area schools were 
similar to those of teachers in the matched traditionally- 
constructed schools used in this study. 

3. There were no consistent differences in student achievement. 

4. There were no consistent differences in student attitudes. 

5. Teachers in open-area schools reported a more open 
organizational climate than did teachers in the matched 
traditionally-constructed schools used in this study. 
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Recommendations of Evaluator: 



1. Continue to build open-area schools. 

2. Encourage additional research on the effects of open- 
area schools on student achievement and attitudes. 



Addi tional Inf orma tion ; 

72- 93 Evaluation Design for Open- Area Schools: 19 72-73, Gary 

W» Womble, 22 pp. 

73- 134 Evaluation Report: Open-Area Schorlj, 1973, Gary W. Womble, 

43 pp. 



Clinton C. Schuhmacher 

Director - 
System-Wide Evaluation 



MEASUREMENT PROFILES 



The Dallas Independent School District Measurement Profiles, 1972-73, 
are a series of six reports that present a comprehensive summary of the 
results of the District's system-wide standardized group-testing program 
as well as selected community socioeconomic indicators. They contain 
summary statistics and norm scores within each grade, sex, and ethnic 
background that present a quantitative overview of measured achievement, 
aptitude, and observed socioeconomic status for each school in the District 
and for the District as a whole. Information contained in the Measurement 
Profiles includes for each school and geographic attendance area: 

1. Median family income and District percentile norm. 

2. Median parental educational level and District per- 
centile norm. 

3. Average daily attendance, average daily membership, the 
ratio of attendance to membership, and District percentile 
norm. 

4. Raw score mean, median, and standard deviation, national 
percentile norm, and District percentile norm for the 
Metropolitan Readiness Test administered in the fall of 
grade one. 

5. Raw score means, medians, and standard deviations, national 
grade-equivalent norms and percentile norms, large-city grade- 
equivalent norms and percentile norms , and District percentile 
norms on all District achievement tests. These include the 
California Achievement Test in the second grade, the 
Comprehensive Tests of Basic Skills in grades three through 
eight, and the Iowa Tests of Educational Development in 
grades nine and twelve. 

6. Language, Non-Language, and Total deviation aptitude scores 
as measured by the California Short-Form Test of Mental 
Maturity, as well as national and District percentile norms. 

The information contained in this series of reports should be 
useful to District decision-makers and the community in a number of 
ways. First, it demonstrates commitment to the school system's policy 
of accountability to parents and the public through a systematic report- 
ing of the educational processes and products of the District. Second, 
it identifies general areas of strength and weakness in the District's 
educational program, thus enabling decision-makers to plan more adequately 
for future programs. Finally, it provides a baseline against which to 
compare the data of past and future years. 
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Important generalizations that can be made from the 1972-73 Measure- 
ment Profiles include: 



1. The comparison between 1971 and 1972 achievemenc results is 
difficult because, due to pronounced population turnover, the 
two groups of students tested are not fully comparable. 

2. When population turnover is taken into consideration, there 
is no evident trend in District achievement levels. That is, 
no practically significant overall increase or decrease exists. 
For instance, in several schools where Anglo turnover is 
minimal, achievement is up in some cases and relatively 
constant in others. In schools that have experienced a large 
loss of Anglo students, the achievement average is lower. 
Thus, because of the problem of population loss, and the fact 
that those students measured at a given grade level in 1971 
are not the same students measured in 1972, the only true 
comparison would be to follow individual students from year 

to year. 

3. Student achievement in the District is highly correlated 
with aptitude, race, parent's income and educational level, 
and, to a lesser extent, sex. This result is consistent 
with the results of similar reports from all parts of the 
nation. 

A. The degree of achievement discrepancy on all tests is 

related to socioeconomic status as indicated by parental 
educational level and family income. The socioeconomic 
data were drawn from the 1970 United States census figures. 

5. l^aen economically deprived minority children enter the 
first grade in the District, they are about a standard 
deviation below the Anglo mean, or in the lower third of 
the distribution on most subtests of the Metropolitan 
Readiness Tests. The degree of discrepancy is related 

to socioeconomic status as indicated by parental educational 
level and family income. 

6. The Anglo versus minority differential in achievement is 
larger in the upper grades than in the lower grades. 

7. Schools in the northwest quadrant of the city scored higher, 
on the average, than schools in other areas. Schools in 
the northwest quadrant scored at about the seventieth 
percentile on local norms , those in the southwest quadrant 
at about the fiftieth percentile, those in the southeast 
quadrant at about the sixtieth percentile, and those in 

the Inner City at about the twenty-fifth percentile. The 
socioeconomic status of the four areas, as indicated by 
parental educational level and family income, conforms to 
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a pattern similar to that observed in ^[achievement scores. 
This underscores the high relationship between socio- 
economic status and achievement. 
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8. Anglo students, on the average, usually scored 'between the 
forty-fifth and sixtieth percentiles on large-city norms, 
Negro students between the fifth and twentieth percentiles, 
and Mexican American students between the tenth and thirtieth 
percentiles. Obviously, the results vary from school to school. 

9. There is marked variability among individual District schools 
relative to measured achievement. This variability can 
generally be tracked along student aptitude, ethnic and 
socioeconomic differences. 

The tests mentioned in this document are tests of generalized 
achievement. It is difficult, short of teaching the test, to show major 
practical differences on these tests in one year's time. The tests 
measure the r lias Independent School District's general educational 
goals, but not specific u.nstructional objectives. For example, a specific 
instructiona] objective that one might be expected to accomplish in a 
given period of time would be "given an orally presented word from the 
Basal Word List, a student will recognize its printed form,'^ Generalized 
achievement in reading is, however, a more long-range process. 

These data may most effectively be used by policy makers in planning 
and implementing improvements in the Dallas Independent School District's 
educational program. For example, if measured achievement in Language 
is low in a given area of the city. District decision-makers can be 
made aware of the need and plan remediation and special programs in 
that subject area. They are descriptive data. They cannot be used 
to explain why a situation exists, only that it does exist. Because 
they are descriptive data, comparisons between schools should be made 
with the greatest degree of care. Socioeconomic differences are 
documented, however, there may be numerous other variables that affect 
student achievement that are not documented in the Profiles . Even with 
the aforementioned shortcomings, the Board of Education is to be commended 
for taking a major step toward joining accountability in education. 



Additional Information : 

72- 88 Design for the 1972-73 Measurement Profiles, Robert L. Mendro, 6 pp. 

73- 127-1 

Through Dallas Independent School District Measurement Profiles, 1973, 
72-127-6 Robert L. Mendro, 

73-127-1 381 pp. 

73-127-2 396 pp. 

73-127-3 390 pp. 

73-127-4 420 pp. 

73-127-5 154 pp. 

73-127-6 76 pp. 
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DALLAS INDEPENDENT SCHOOL DISTRICT 
TEACHER PROFILES 



The Dallas Independent School District Teacher Profiles, 1972-73 > 
are a series of three reports that present a summary of selected 
academic and demographic characteristics of District classroom teachers. 
They should be studied in conjunction with the Measurement Profiles in 
order to obtain an accurate picture of the status of education in the 
Dallas Independent School District (DISD) . Information contained in the 
Teacher Profiles includes for each attendance area: 



1. 


sex of teachers 


2. 


age distribution of teachers 


3. 


marital status of teachers 


4. 


ethnic background of teachers 


5. 


state of birth of teachers 


6. 


teacher educational level 


7. 


teacher experience 


8. 


teacher certification 


9, 


percentage of teachers teaching in major area of 




academic preparation 



Important generalizations that can be made from the 1972-73 Teacher 
Profiles include: 

1. DISD faculty is racially balanced. When considering 
Anglo and Negro teachers, racial balance is generally 
maintained within, as well as across, District schools. 
However, only about 2% of the District's teachers are 
Mex i c an- Amer i c an . 

2. There is no systematic bias in teacher assignment 
relative to academic background, that is, schools 
are not significantly different in the academic 
backgrounds of their staffs. 

3. The vast majority of DISD teachers attended a Texas 
college or university. The major suppliers, in order, 
are North Texas, East Texas, Bishop, Prairie View, 
and Southern Methodist University. 

l\. Some District schools tend to have older, more 
experienced staffs than others. 

5. On the average, junior high teachers are the 
youngest group, elementary teachers, the oldest, 

6. In comparison with females, males are: 

a) older 

b) more likely to be married 

c) more likely to hold a Master's degree 
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Teacher scores on the National Teacher Examinations 
were not reported because over 50% of District 
teachers do not have such scores on record. 



Teachers of all races are equally likely to be 
teaching in their academic area of major preparation. 

There is an overwhelming preponderance of female 
teachers at the elementary level. Sex of teachers 
is more evenly distributed at the secondary level. 



Additional Information : 

72-170 Dallas Independent School District Teacher Profiles 
Elementary Schools, 1973, Fredell H» Pollak, 31 pp. 

72-171 DallaF Independent School District Teacher Profiles 
Junior High Schools, 1973, Fredell H. Pollak, 14 pp. 

72-172 Dallas Independent School District Teacher Profiles 
Senior High Schools, 1973, Fredell H. Pollak, 12 pp. 
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A PRELIMINARY SURVEY OF THE RELATIONSHIP BETWEEN SOCIOECONOMIC 
STATUS AND ACHIEVEMENT FOR DALLAS PUBLIC ELEMENTARY 
SCHOOL CHILDREN 



This study was designed to investigate the question: 

Within ethnic background, do achievement and 
aptitude scores of Dallas elementary school 
children track according to their socio- 
economic status? 

The investigation was severely limited by the fact that socio- 
economic data were only available by school community rather than by 
individual student. Subjects vere all students enrolled in grades 
one to six in the Dallas Independent School District during the 1972-73 
school year for whom scores were available on at least one of the 
District's system-wide aptitude or achievement tests administered during 
the fall of that year. The index of socioeconomic status used was 
drawn from census figures on median parental income and educational level 
for each school community. 

The results of the study suggested that: 

Within ethnic background, student aptitude and 
achievement scores were indicative of the 
socioeconomic level of the school community. 
That is, the higher the socioeconomic level, 
the higher the student aptitude and achievement 
s cores . 

Socioeconomic status was used as a measure of home environment and, 
thus, was thought to be indicative of a child's opportunities for intellec- 
tual growth in the home. The environmental portion of achievement is 
amenable to intervention. The focus of the study was on socioeconomic 
status as an indicator of: 

1 . achievement press 

2. language models in the home and peer group 

3. academic guidance provided in the home 

4. stimulation to explore various aspects of the 
larger environment provided in the home 

5. intellectual interests and activities in the 
home and peer group 

6. work habits emphasized in the home and in the peer group 
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Thus, socioeconomic status is highly related to achievement among 
Dallas elementary school children. This further suggests that appro- 
priate environmental intervention in the home could provide a pronounced 
contribution toward reducing the achievement gap and providing quality 
education for all studnets. In order for valid conclusions to be 
reached relative to the effects of home intervention on student achieve- 
ment, an experimental program, involving programmatic intervention in the 
home environments of poor children, with a valid experimental design^ 
would have to be implemented. 

The tentative nature of the results reported in this study cannot be 
overemphasized. This is an exam.ple of ex post facto cross-sectional re- 
search. What is important for the reader to understand is that achieve- 
ment has two basic classes of components: those that the schools can 
modify and those that they cannot. It is essential that educators and 
parents alike understand that there is some portion of a student's aca- 
demic achievement, regardless of his ethnic background, that is amenable 
to environmental intervention. It is important to realize, however, that 
only limited types of environmental manipulation are currently open to 
educators . 



Additional Information : 

72- 86 Socioeconomic Profile of School Communities in the Dallas 

Independent School District, 1972-73, Charles A. Hunter, 
23 pp. 

73- 181 A Preliminary Survey of the Relationship between Socio- 

economic Status and Achievement for Dallas Public Elementary 
School Children, 1973, William J. Webster and Robert L. Mendro. 
35 pp. 
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FIVE-YEAR ENROLLMENT TRENDS, 1968-1972 



This report examines enrollment data for District schools since 
1968. Some attention is devoted also to consideration of suburban 
enrollment statistics in order to facilitate interpretation of the 
DISD patterns. Following are the principal observations noted in the 
text of the report: 

1. Anglo enrollment since 1968 showed a slight downward 
trend which was accelerated markedly following the 
court decision of 1971. 

2. Negro and Mexican- American enrollments have increased 
at a fairly steady rate since 1968. 

3. Within the District, the most rapid population shifts 
have occurred in Oak Cliff, the slowest in North 
Dallas. The same patterns of change, however, have 
been observed in all areas. 

4. Most of the Anglo students lost in 1971-72 and 1972-73 
apparently either enrolled in private schools or moved 
out of Dallas county. 



Additional Information ; 

73-192 Five-Year Enrollment Trends, 1973, Clinton C. Schuhmacher, 
41 pp. 
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INCIDENCE OF DROPOUT 



This study was undertaken primarily to determine the extent of the drop- 
out problem during 19 71-72 in the Dallas Independent School District. 
Attempts were made to identify reasons for students leaving school 
voluntarily before graduation, to identify District dropout character- 
istics, and to provide a progress report for the Metropolitan Alterna- 
tive School System. In addition, an attempt was made to determine if 
the dropout problem was more critical in certain high schools and/or 
ethnic classifications • 

Major results are as follows: 

1» The system-wide dropout rate for the District's high schools 
was reported as 11.28%. Dropouts were defined as those 
students who left school and were unaccounted for during the 
twelve-month period from September, 1971, to August, 1972. 

2. This is the first year that the Department of Research and 
Evaluation has undertaken a dropout study. Previous studies 
defined dropout differently. The most recent dropout study 
completed reported a system-wide 1970-71 dropout rate at the 
senior high school level of 5.6%. A comparable definition for 
dropouts would have produced a 1971-72 dropout rate of 5.31%. 

3. Dropouts were reported for each District high school for the 
1971-72 school year. Generally, predominantly Negro high 
schools from lower socioeconomic areas reported higher drop- 
out rates, while predominantly Anglo high schools from higher 
socioeconomic areas reported lower dropout rates than other 
District high school's. 

4. System-wide Negro and Mexican-American children evidenced 

a higher dropout rate than did Anglo children. Tlie American 
Indian dropout rate was nearly twice that of any other ethnic 
classification. Oriental children evidenced the lowest drop- 
out rate. 

5. "Work" and "marriage and/or pregnancy" were the two major 
meaningful school-reported reasons for students dropping out 
of school in 1971-72. 

6. There does not appear to be a significant difference between 
the dropout rate for females and the dropout rate for males. 

7. The growth of the Metropolitan Alternative School System and 
its relatively low dropout rate are encouraging. Apparently, 
this system is experiencing success helping dropout-prone 
students to attain educational success. Approximately four 
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out of five dropout-prone students are being retained through 
the Metropolitan Alternative School System. 

In the text, methodologies are suggested for future dropout studies. 
In addition, a list of reasons suggested for use by students withdraw- 
ing from school is provided in Appendix C. Further recommendations for 
future reports are as follows; 

1. Place a major emphasis on predominantly Negro and inner-city 
schools . 

2. More time should be devoted to the evaluation of the Metro- 
politan Alternative School System. 

3. District decision-makers should consider the adoption of a 
policy regarding the number of days a student may be absent 
before his name is permanently dropped from the school roster 
and/or an effort is made to locate him. 



Additional Information : 

73-143 Incidence of Dropout During 1971-72, Ronald S. King, 90 pp. 
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GRADUATE FOLLOW-UP 



Description of the Study ; In October of 1972, all Dallas high 
schools participated In a telephone survey designed to ascertain 
the vocational and educatlonaf^pursults of their recent graduates. 
This study marked the third year of graduate follow-up surveys. 

Results: The results of the survey were categorized by sex within 
District schools. The following figure best summarizes the data 
available. 



1970 Graduates 



1971 Graduates 



1972 Graduates 




ing Attending 
4-year Junior 4-year 
College College and Jr. 

College 



ing tary 
Business Serv- 
or Tech- ice 
nlcal 
School 



time 1 all- 
House- Tim. 
wives 



* J£m]^Loyea linempioyed 
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Additional Information : 

72-121 Follow-up of 1972 Graduates. Clinton C, Schuhmacher and 
Kay Maxwell, 7 pp. 
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SYSTEM-WIDE DRUG SURVEY RESULTS 



Description of the Study 

The system-wide drug survey reports the results of the fifth large- 
scale survey of drug usage as reported by Dallas public school children. 
The results represent the tabulation of a 10% stratified random 
sample of Dallas public school children. Since data were available from 
four previous surveys, trends in drug usage were also traced over time. 
Additionally, the data were tabulated by grade, sex, ethnicity, geographic 
area, and academic performance. This report provides District decision- 
makers with information about the extent and concomitants of drug usage 
among public school children. 



Results 

1. Reported drug usage increased generally in grades nine 
through twelve since the 1972 drug survey. Trends were 
mixed, however, in grade eleven and in the South Dallas- 
Kleberg-Seagoville area. System-wide trends in reported 
drug usage are summarized below. 



Directions of Changes between 
1972 and 1973 Usage Data 



Drug 


Grades 9-12 


Grades 5-8 


Alcohol 


Stable 


Down slightly 


Tobacco 


Stable 


Stable 


Marijuana 


Up sharply 


Up slightly 


Hashish 


Up slightly 


Stable 


Inhalants 


Stable 


Down slightly 


Prescription stimulants 


Stable 


Stable 


Prescription sleeping pills 


Stable 


Down slightly 


LSD 


Up slightly 


Stable 


Mescaline and peyote 


Up 


Stable 


Misc. narcotics 


Up 


Stable 


Cocaine 


Up 


Stable 


Prescription tranquilizers 


Stable 


Stable 
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2. In grades five through eight, the change, although less 
pronounced, was generally in the direction of a de- 
crease. Some increases, however, were observed, par- 
ticularly in North Dallas and Southwest Oak Cliff. 

3. Relative to other drugs, the data on marijuana reflect 
the largest increases. 
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Reported usage in North Dallas and, to a lesser extent, 
Southwest Oak Cliff was higher than in other areas. 



5. There was little evidence of a relationship between 
reported drug usage and ethnicity, except for the 
relatively high percentage of Negro respondents classi- 
fied as users in grades eleven and twelve. 

6. Males showed much more inclination to use drugs than 
did females. 

7. The use of drugs is strongly and inversely correlated 
with academic performance. 



Additional Information ; 

73-132 System-Wide Drug Survey Results, 1973, Clinton C. Schuhmacher , 
32 pp. 
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INDUCED DESEGREGATION: ITS EFFECTS ON 
STUDENT ACHIEVEMENT AND 
POPULATION RESEGREGATION 



Recently, heated debate has erupted in the social science lit- 
erature concerning the effects of induced desegregation and its most 
visible tool, forced bussing, on student achievement and associated 
variables. The proponents of bussing have cited research which they 
have claimed has demonstrated positive effects of induced desegrega- 
tion and bussing, while bussing' s opponents have claimed opposite 
results. The situation has understandably caused a great deal of 
public and professional confusion about the effects of bussing and 
induced desegregation. 

The paper had two primary purposes. First, in light of the 
confused state of desegregation research, a representative group of 
studies on the effects of induced desegregation, including many of 
the reports that are featured most prominently in the current debate, 
was reviewed in an attempt to explain some of the factors contrib- 
uting to the contradictory state of recent desegregation dialogue. 
Second, because of rapidly increasing proportions of minority popu- 
lation in most major urban areas, the relationship between court- 
ordered desegregation and population instability in large school 
systems was examined particularly as such instability related to 
Anglo enrollment loss. The second phenomenon has been largely ig- 
nored in studies examining the effects of induced desegregation. 

In order to facilitate the examination of the research, a thor- 
ough review of methodological techniques appropriate for examining 
the eftects of induced desegregation was presented. It included 
necessary design prerequisites, statistical analysis techniques, 
reporting standards, and minimum requirements and precautions when 
recommended design factors cannot be employed. Studies of fifteen 
induced desegregation programs were examined using the standards 
specified in the methodological review. The analysis clearly dem- 
onstrated that, in general, the existing literature suffered from 
many faults both in terms of experimental design and in terms of 
reporting standards. Particular problems included inadequate de- 
scriptions of treatments, subjects, classroom procedures, desegre- 
gation procedures, and curriculum implementation; inadequate data 
on the comparability of treatment groups; incomplete reporting of 
necessary statistical data; and a nearly total lack of process 
evaluation. 

The examination of the relationship between enrollment patterns-, 
and desegregation orders in large-city school systems clearly demon- 
strated a marked correlation between such orders and Anglo enroll- 
ment loss. Although correlation does not necessarily imply causation, 
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the relationship between desegregation orders and abrupt declines in 
Anglo enrollment was too conspicuous to be attributed to chance. 
The relationship was particularly strong at the elementary level. 

Thus, two major generalizations emerged from the data analyzed 
in the paper: 

1. Reliable evidence pertaining to the success or failure 
of induced desegregation in general, and bussing in 
particular, in affecting student achievement and asso- 
ciated variables does not exist. The only statement 
that appears warranted by existing data is that the 
achievement of neither majority nor minority students 
seems to be adversely affected by induced desegrega- 
tion. No conclusive evidence concerning possible 
positive effects of induced desegregation on minority 
achievement was presented in the studies reviewed. 

2. If the major purpose of desegregating the public 
schools is to promote societal integration, then 
current methods are failing In the large cities. 
The cities and their public school systems are 
being resegregated at an alarming rate. 

Before completing the Executive Summary, and in light of con- 
siderable misinterpretation of an earlier draft version of part of 
the paper, the authors feel it necessary to discuss three limita- 
tions of the paper: 

1. It is not a report of original research done by the 
Dallas Department of Research and Evaluation. It is 
a thorough review of studies done elsewhere by re- 
searchers other than Dallas researchers. 

2. The paper in no way questions the validity of the 
Supreme Court decision of 1954. The purpose of 
the paper was to examine several effects of in- 
duced desegregation as it is currently being 
implemented. The fact that such implementation 
was found to be unsatisfactory in light of the 
evidence on resegregation does not imply that 
desegregation itself is undesirable. Such a 
decision would have to deal first with the ex- 
tent to which moral and ethical issues affect 

the question of desegregation. The data in 
the study have absolutely no relevancy to that 
que St ion • 
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3. The paper is neither pro- nor antibussing. The re- 
search on induced desegregation does not deal with 
bussing alone. If it were possible to draw valid 
conclusions concerning the adequacy of induced de- 
segregation as an educational treatment, given exist- 
ing data, such conclusions would be valid only for 
entire desegregation programs, not bussing alone. 
Since the paper clearly demonstrates the inappropriate- 
ness of drawing these types of conclusions, any pro- 
bussing or antibussing interpretations of the data 
in the study must be supported primarily on the emo- 
tions and prejudices of the reader. 

What the review does suggest is that valid consistent evidence 
does not exist to support the contention that recent court decisions, 
as they have been implemented by public schools and their associated 
communities, have substantially contributed to increasing the quality 
of education for minority students; and that valid consistent evi- 
dence does exist in the case of large cities to support the conten- 
tion that such decisions and their associated implementation have 
contributed to rapidly increasing resegregation of the public school 
population, thus thwarting the goal of meaningful integration. 

The data suggest that the time has come to examine current policies 
and procedures designed to end racial isolation and discriminat nn. The 
problem of racial isolation in the cities requires innovative solutions 
that are not counterproductive. Such soiutions might involve expanding 
the geographic base for desegregation, partial desegrep,ation involving 
advanced educational technology and/or the maintenance of specific 
majority-minority ratios, part-time desegregation for ccirtain types 
of activities involving equal status contact on neutral turf, the 
construction and use of educational parks, '^r coiabinatlons ol the 
above. The time for trying nev;, innovative approaches to stimulating 
societal integration through the public schools is lun;. 

Additional Information : 

73-204 Induced Desegregation: Its Effects on Student Achievement and 
Population Desegregation, 1973, William J. Webster and Robert 
L. Mendro, 70 pp. 



SOCIOECONOMIC PROFILES OF SCHOOL COMMUNITIES IN THE 
DALLAS INDEPENDENT SCHOOL DISTRICT 



Objiectives of the Study ; The overriding objective of this study 
was to establish a data base of socioeconomic characteristics of 
the populations within the boundary of each school community as a 
f rame-of-ref erence for evaluating achievement, planning and exe- 
cuting programs, and assessing particular behavior of pupils. Such 
a data base of socioeconomic characteristics will permit adminis- 
trators and teachers to relate the teaching-learning experience 
more relevantly to each student and to plan more meaningful social 
contacts, among students and teachers. 

Samplr:: ; The report utilized the U.S. Census data of social charac- 
teristics of the population of Dallas as a source of information. 

Evaluation Design ; The particular census tracts which fell into 
each school community were grouped tO'^,2ther to form individual units 
of information, yielding median years of education completed, age of 
population, level of housing valuation, and median family income. 
From these data, it was possible to categorize segments of each school 
population as to dominant sociocultural influences and socioeconomic 
status . 



Evaluation Results : Preliminary results show some consistent patterns 
of income and educational levels of populations in proximity to one 
another as well as differential patterns among distant communities. 
These results show that there are vast differences to be observed 
between the lowest socioeconomic status level and the highest socio- 
economic level in the District. Such information can be useful in 
designing programs for pupils who represent wide ranges of differ- 
ences in a single setting. 



Additional Information : 

72-86 Socioeconomic Profiles of School Communities in the Dallas 
Independent School District, 1972, Charles A. Hunter, 5 pp. 

72-92 Socioeconomic Profiles of School Communities in the Dallas 

Independent School District, 1972, Charles A. Hunter, 23 pp. 

Socioeconomic Profiles of School Communities in the Dallas 
Independent School Distict, 1973, Charles A. Hunter 
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SOCIOECONOMIC PROFILES OF SCHOOL COMMUNITIES 
IN THE DALLAS INDEPENDENT SCHOOL DISTRICT 
PART II 



Objectives of the Study ; The Socioeconomic Status Level Study pro- 
vides a data base which will permit the development of a community 
profile of social and demographic characteristics. Such data will 
be useful in evaluating the educational products of the schools and 
serve as a basis for decision-making on the part of administrators. 

The Design ; This report, the second of two, utilizes U.S. census 
data that are gathered in components represented as school conmiuni- 
Jties. These school communities are determined by the boundaries of 
each elementary school in the District and identified by the name 
of that school. Frequency distributions of the characteristics of 
the population in these comiiiunities are presented in tabular form. 
Variables of family income, occupational distribution, family type, 
home ownership status, housing valuation, and school dropouts are 
dealt with. Characteristics of individual school communities are 
noted and comparisons made between school communities. 

Evaluation Results ; The results show a distinct observable differ- 
ance between particular communities as to occupational characteristics, 
educational range, housing valuation, and to a lesser extent, family 
types. There tended to be little integration of the characteristics 
within particular communities. VJhere major differences occurred, they 
were between communities* An occupational ranking scale was used to 
denote the occupational characteristics of school communities. Five 
general categories have been delineated, including 1) Professional, 
2) Managers, 3) Sales and Clerical, 4) Craftsmen, Operatives, and 
Transportation workers, and 5) Labor, Service, and Household workers. 
For convenience, the scale has been described in three intervals, 
namely, upper level (white collar), middle range (generally blue collar), 
and lower level (labor and service workers). Seventy- four of the 128 
school communities fall in the lower level of the scale while two fall 
in the middle range. Twenty-nine of the 128 school communities are 
occupationally integrated, that is, the types of occupations are gener- 
ally distributed throughout the scale. The study shows a high percent- 
age of women in the work force in Dallas, but, at the same time, they are 
under-represented in the professional and managerial occupations. Women 
tend to fall more in the lower economic level as there is a high corre- 
lation between the communities with higher percentages of female-headed 
households and those that fall toward the lower level of the occupational 
scale. The relationship between income and education is not comparable 
in predominantly black communities and predominantly white ones, since 
a rise in educational level among blacks does not result in similar 
rises in income levels as is true of whites. 
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The study examines the frequency of family types in the District and 
shows that certain communities have higher percentages of families 
with female-headed households than is true of most school communities. 
Those communities with more female-headed households tend to fall in 
the lower socioeconomic level. Home ownership tends to have a high 
frequency throughout the DISD. The majority of those communities 
with low frequency of home ownership are, for the most part, communities 
around Title I schools. 



Recommendations : 

1. that each school faculty be requested to acquaint 
itself with the data presented in the report so 
that the faculty may create a frame of reference 
for the pupils of the school, creating a better 
understanding of the behavior of both pupils and 
faculty. 

2. that each teacher use the profile of his particular 
school community to evaluate the behavior, performance^ 
and interaction of his/her pupils. 

3. that the Socioeconomic Status Level Study become a 
basic document in the DISD to be used as a reference 
for all evaluations made of pupil achievement and 
pupil behavior in the schools. 

Additional Information; 



72-86 



Socioeconomic Profiles of School Communities in 
the Dallas Independent School District, 1972, 
Charles A. Hunter, 5 pp. 



72-92 



Socioeconomic Profiles of School Communities in the 
Dallas Independent School District, .1972, 
Charles A. Hunter, 23 pp. 



73-196 



Socioeconomic Profiles of School Communities in the 
Dallas Independent School District, 1973, 
Charles A. Hunter 
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A SYSTEM-WIDE ASSESSMENT OF INDIVIDUALIZED INSTRUCTION: 
VOLUME I - OVERVIEW AND SUMMARY 



Description of the Study ; Assessment of the ejctent of individualization was 
identified as a priority area for evaluation by both the Elementary and 
Secondary Departments of the Dallas Independent School District's Operations 
Division; therefore, a major study (involving some 2,100 teachers and 2^00 
students) was undertaken in order to provide the desired information. The 
full report of that research effort consists of a series of six volumes. 
The first volume served primarily as an introduction, outlining procedures 
of sampling and instrumentation while presenting results in very general 
terms. 



Results ; At the risk of oversimplification of data subtleties, some of the 
major results are sketched below. The reader is strongly urged, however, 
to refer to the individual volumes (or at least the summaries) , since the 
brief list which follows can provide, at best, only a superficial under- 
standing. 

1. Administratively, the District had moved toward increased 
individualization by attempting to decentralize decision- 
making, by providing numerous alternative programs (e.g., 
career development programs, developmental programs, etc), 
and by supporting staff development in areas related to indi- 
vidualization. There were, however, a number of areas in 
which official policy inhibited individualization; among those 
were scheduling, grading practices, and criteria for pro- 
motion and graduation. 

2. Teachers generally seemed to have some understanding 
of the concept of individualization and were able to 
identify components specified as important in the 
literature. There was some evidence, however, that 
teachers needed increased training in dealing with 
individual differences among students. 

3. Classroom practices were often found to conflict with the 
recommendations of the literature on individualization. 
Particularly was this true in regard to (a) limited use of 
behavioral objectives, (b) failure to provide varying 
sets of objectives, (c) little use of available methods 
and materials, and (d) failure to evaluate students in 
terms of specific, objective criteria. 

Recommendations : In terms of standards abstracted from the literature, it 
seemed clear that the District was not substantially individualized, leather 
those standards were appropriate, however, was open to question, consider- 
ing that the literature tended to be somewhat idealistic and not altogether 
consistent. The fact emerged that the District had not specified its own 
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operational definition of individualization, despite what seemed to be a wide- 
spread sentiment in favor of the concept. The major recommendation emanating 
from the study, therefore, was that the District explicitly identify the 
aspects of individualization which were to be considered desirable and attain- 
able. If components not currently present in the Instructional program 
of DISp were so identified, then a detailed plan of action should be formulated 
and implemented. 



Additional Information: 



72- 109 Evaluation Design for the System-Wide Assessment of Indi- 

vidualization, 1973, George H. Olson, 85 pp. 

73- 161-1 A System-Wide Assessment of Individualized Instruction: 

Volume I - Overview and Summary, 1973, George H. Olson, 
35 pp. 

73-161-2 A System-Wide Assessment of Individualized Instruction: 

Volume II - Review of Related Literature, 1973, 
George H. Olson, 56 pp. 

73-161-3 A System-Wide Assessment of Individualized Instruction: 

Volume III - Administrative Considerations, 1973, 
George H. Olson, 29 pp. 

7 3-161-4 A System-Wide Assessment of Individualized Instruction: 

Volume IV - Classroom Practices - Elementary and Secondary, 
1973, George H. Olson, 106 pp. 

73-161-5 A System-Wide Assessment of Individualized Instruction: 

Volume V - Teacher Characteristics and Considerations, 
1973, George H. Olson, 30 pp. 

73-161-6 A System-Wide Assessment of Individualized Instruction: 

Volume VI - Appendices, 1973, George H. Olson, 66 pp. 
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A SYSTEM-l^IDE ASSESSMENT OF INDIVIDUALIZED INSTRUCTION: 
VOLUME II - REVIEW OF RELATED LITERATURE 



Description of the Study ; One of the major problems encountered at the onset 
of the survey of individualized instruction in the DISD was the problem of 
definition. In an attempt to arrive at a definition that would meet the 
requirements of objective raeasurenient , most of the literature relating to 
the nature of iriHividualized instruction itself was reviewed. Volume Two 
represented the rcGults of that review. 

Results : The review briefly highlighted the history of individualized 
instruction from the early colonial days to the mid-1900's. In this 
development, the modern-day push toward individualization was cast as 
evolving from a need to create a mure responsive educational system. Where 
earlier forms of education were directed more along the lines of producing 
efficient education for the masses, new forms of education were seen aJ=; 
being responsive to individual differences among learners. 

The attempts to define individualized instruction within the literature 
were considered next. Articles and books dealing with the goals and 
specific definitions of individualization were reviewed. It was concluded 
that individualization was, at best, a highly amorphous concept. The only 
general definition that could be advanced was that individualized instruction 
was "the adaptation of the instructional environment to individual differ- 
ences." 

A variety of approaches which had traditionally been employed to adapt 
instruction to the variability that existed among learners was then 
reviewed, and many were criticized, for one reason or another, as to 
meeting the needs of individual learners. Included among these approaches 
were acceleration and retardation, enrichment and remediation, ability 
grouping, departmentalization, and the notion of the self-contained class- 
room. The review was followed by a brief consideration of many of the 
new formal and informal approaches that had been proposed to more or less 
"tailor" instruction to individual differences. 

Careful review of the literature had failed to provide a working definition 
of individualized instruction suitable to meet the needs of the DISD sur- 
vey; therefore, the literature was again reviewed for the purpose of 
extracting those instructional components which enjoyed wide acceptance 
among educational authorities as being important to an individualized system. 
The second was more fruitful in that it yielded a list of instructional 
components, the existence of which potentially could be determined. These 
components spanned a range from recommended administrative requirements, to 
classroom instructional requirements, to required teacher characteristics. 
These required components of individualization, in turn, served as the 
variables of analysis in the study of individualization in the DISD. 
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The major features identified from the literature as characteristic 
of an individualized system are the following: 

Administration 

Curriculum Goals 

Multipath Curriculxim 

Alternative Instructional Strategies 

Commitmetit to the Attitude and Philosophy of 

Individualization 
Research and Development 
Computer Utilization 
Decentralized Decision-Making 
Ipsative Grading Practices 
Flexible Scheduling 

Classroom 

Operational Objectives 
Variability in Choice of Objectives 
Variability in Sequence of Objectives 
Criterion-Referenced Measurement 
Variability in Achievement Criteria 
Diagnosis of Entering Ability 
Continuous Assessment 

Variability ox Instructional/Learning Methods 

Variability in Rate of Progress 

Student /Teacher Consultation and Guidance 

Emphasis on Student Declsion^Making 

Resource Centers 

Flexible Grouping Practices 

Arrangement and Availability of Educational Resources 
Teacher 

Sensitivity to Indiv^-dual Differences 
Acceptance of Individual Differences 
Utilization of Student Characteristics Data 
Guidance and Counseling 
Diagnosis and Assessment 
Individual, Pupil-Teacher Contact 
Role in Decision-Making 
Media Utilization 
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Additional Inf orntation; 



72- 109 Evaluation Design for the System-Wide Assessment of Indi- 

vidualization, 1973, George H, Olson, 85 pp, 

73- 161-1 A System-Wide Assessment of Individualized Instruction: 

Volume I - Overview and Summary, 1973, George H, Olson, 
35 pp, 

73-161-2 A System-Wide Assessment of Individualized Instruction: 

Volume II - Review of Related Literature, 1973, 
George H. Olson, 56 pp, 

73-161-3 A System-Wide Assessment of Individualized Instruction: 

Volume III - Administrative Considerations, 1973, 
George H. Olson, 29 pp. 

73-161-4 A System-Wide Assessment of Individualized Instruction: 

Volume IV - Classroom Practices - Elementary and Secondary, 
1973, George IK^Glson, 106 pp- 

73-161-5 A System-Wide Assessment of Individualized Instruction: 

Volume V - Teacher Characteristics and Considerations, 
1973, George H, Olson, 30 pp. 

73-161-6 A System-Wide Assessment of Individualized Instruction: 

Volume VI - Appendices, 1973, George H. Olson, 66 pp. 
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A SYSTEM-WIDE ASSESSMENT OF INDIVIDUALIZED INSTRUCTION: 
VOLUME III - ADMINISTRATIVE CONSIDERATIONS 



Des c ription of the Study : Before individualized instruction could be expected 
to take place on any large-scale, system-wide basis, there were several 
requirements to be provided which could only be provided by the administration • 
Many of those requirements were described in the previous volume. (Volume II). 

To examine the District's stance in relation to tl-ose requirements, news- 
paper clippings, DISD publications, survey instruments, informal inter- 
views with administrators^ and evaluation reports of the Department of 
Research and Evaluation were utilized. From these sources of information, 
it was possible to determine, in part, where the District stood in relation 
to the recommendation of instructional authorities. 



Results: Among the general conclusions obtained from the investigation were 
the following: 

1. The District had taken definite steps toward * 
decentralization of educational decision-making. 

There were^ nevertheless, a number of areas in 
which official policy inhibited individualization; 
among those were scheduling, criteria for promotion 
and graduation, and grading practices. 

2. The District appeared to emphasize programs of 
instruction that served the most students 
effectively. In doing so, the District 
indirectly supported group-forms of instruction. 

3. The District had made attempts to educate at least 
part of the faculty in methods of individualized 
instruction. 

4. The District had made important contributions in an 
often-overlooked area of individualization. Through 
its many alternative educational programs (e.g. career 
d'^velopment programs, developmental programs, etc.), 
the DISD had sought to adapt a significant portion 

of its instructional offerings to the unique needs 
of special groups of individuals. 

Recommendations : Ultimately, it could not be concluded that the DISD 
was significantly individualized. Too many constraints on flexibility 
and adaptation existed. Whether those constraints were necessary for a 
school system the size of the DISD was not investigated in the study. 
Nevertheless, before it could be said that DISD was an individualized 
school system, changes in administrative policy to bring the total realm 
of educational decision-making closer to the student would have to be made. 
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A significant requirement, assuming that further individualization was 
desired, was a formal conrniitinent and plan for individualizing the District's 
instructional program. In the investigaLion, no detailed, coordinated 
plan for achieving that form of instruction could be identified. 



Add! t iona I In forma t ion : 



72- 109 Evaluation Design for the System-Wide Assessment of Indi- 

vidualization, 1973, George II. Olson, 85 pp. 

73- 161-1 A System-Wide Assessment of Individualized Instruction: 

Volume I - Overview and Summary, 1973, George H. Olson, 
35 pp. 

73-161-2 A System-Wide Assessment of Individualized Instruction: 

Volume II - Review of Related Literature, 1973, 
George H. Oldon, 56 pp. 

73-161-3 A System-Wide Assessment of Individualized Instruction: 

Volume III - Administrative Considerations, 1973, 
Cieorge H. Olson, 29 pp. 

73-161-4 A System-Wide Assessment of Individualized Instruction: 

Volume IV - Classroom Practices - Elementary and Secondary, 
1973, George H. Olson, 106 pp. 

73-161-5 A Systein-Wide Assessment of Individualized Instruction: 

Volume V - Teacher Characteristics and Considerations, 
1973, George H. Olson, 30 pp. 

73-161-6 A System-Wide Assessment of Individualized Instruction: 

Volume VI - Appendices, 1973, George H. Olson, 66 pp. 
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A SYSTEM-WIDE ASSESSMENT OF INDIVIDUALIZED INSTRUCTION: 
VOLUME IV - CLASSROOM PRACTICES - 
ELEMENTARY AND SECONDARY 



Description of the Study : In Volume Four, a detailed analysis of the 
results obtained from the many survey and observation instruments 
used in the study was presented. An item-by-item analysis was under- 
taken in order to provide a more in-depth description of the District's 
status with respect to individualization in the classroom. Aspects 
of individualized instruction considered were those involving: 

1. behavioral objectives, 

2. variety and choice of objectives, 

3. variability in instructional methods and materials, 

4. pacing of instruction, 

5. criterion-referenced measures and differential 
achievement levels , 

6. subgrouping, and 

7. assessment of entering ability. 

It should be pointed out that there did not exist an independent 
standard against which the District's teachers' responses to these items 
could be compared; therefore, item responses had to be interpreted in 
an absolute sense. 



Results ; 

1. Although significant proportions of teachers reported 
using behavioral objectives, they did not possess a 
high degree of skill in recognizing the essential com- 
ponents of behavioral objectives. Objectives were not 
generally given to students, and many students indicated 
that they did not know what the intended outcomes of their 
instruction were. 

2. Teachers did not tend to individualize their instruction 
by providing different learners with different sets of 
instructional objectives, although they agreed that the 
procedure was important for individualized instruction. 
Elementary teachers reported being more flexible than 
secondary school teachers. 

3. The majority of teachers did not appear to utilize a 
wide variety of instructional materials and methods. 
The evidence indicated that instruction was generally 
group-orientfid. A wide variety of educational resources 
were available to teachers . 
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4. To a limited degree, teachers allowed students to 
pace themselves through instructional materials. 
However, many teachers indicated that self -pacing 
was allowed only insofar as permitting students to 
proceed at a faster than predetermined rate of pro- 
gress. When queried about the source of decision- 
making with respect to instructional pacing, teachers 
most often identified themselves as this source. 

5. Teachers reported that they set differential 
expected achievement levels for different students 
and that they evaluated students in terms of 
individual learner characteristics . However , 
they did not, as a whole, evaluate students in 
terms of specific, objective criteria. 

6. The practice of subgrouping was widespread in 
elementary schools and evident in secondary schools 
to a considerably lesser degree. However, the 
patterns of subgrouping were not always particularly 
conducive to individualized instruction. 

7. Most teachers apparently believed that the assess- 
ment of entering ability prior to beginning new 
sequences of instruction was an important component 
of individiaalized instruction. Furthermore, many 
teachers reported that they did, in fact, seek 
information regarding students' earlier performance. 



Recommendations ; In order to move toward increased individualization, the 
District would apparently find it necessary to expand its staff develop- 
ment effort, since many teachers -appeared to lack skill in the use of 
behavioral objectives, criterion-referenced tests, etc. Additionally, 
it seemed clear that a commitment would be required of principals and 
other supervisory personnel. It was doubtful that the substantial behavioral 
modifications required of teachers would likely occur in the absence of 
reinforcement from local administration. 

The importance to be attached to the above comments was, of course, 
dependent upon the level of individualization desired for DISD. 



Additional Information ; 

72- 109 Evaluation Design for the System-Wide Assessment of 

Individualization, 1973, George H. Olson, 85 pp. 

73- 161-1 A System-Wide Assessment of Individualized Instruction: 

Volume I - Overview and Summary, 1973, 
George H. Olson, 35 pp. 

73-161-2 A System-Wide Assessment of Individualized Instruction: 

Volume II - Review of Related Literature, 1973, 
George H. Olson, 56 pp. 
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73-161-3 A System-Wide Assessment of Individualized Instruction: 

Volume III - Administrative Considerations, 1973, 
George H. Olson, 29 pp. 

73-161-4 A System-Wide Assessment of Individualized Instruction: 

Volume IV - Classroom Practices - Elementary and Secondary, 
1973, George H. Olson, 106 pp. 

73-161-5 A System-Wide Assessment of Individualized Instruction: 

Volume V - Teacher Characteristics and Considerations, 
1973, George H. Olson, 30 pp. 

73-161-6 A System-Wide Assessment of Individualized Instruction: 

Volume VI - Appendices, 1973, George H. Olson, 66 pp. 
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A SYSTEM-WIDE ASSESSMENT OF INDIVIDUALIZED INSTRUCTION: 
VOLUME V - TEACHER CHARACTERISTICS AND CONSIDERATIONS 
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Description of the Study 

Volume Five was concerned with the teacher characteristics involved 
in individualized instruction. The teacher's role in the decision-making 
process in the classroom was examined in relation to individualzied 
instruction. Another teacher characteristic discussed as an important 
aspect of individualization was teacher sensitivity to individual 
differences. The characteristic was indirectly measured through teachers' 
predictions of the responses their students would make to an attitude- 
toward-school questionnaire. In addition, the attitudinal orientation of 
teachers toward individualization was evaluated. 

Results 

1. A large amount of student decision-making responsibility 
was not evident from the responses of teachers and students 
concerning class practice. However, many teachers in theory 
seemed to support increased student responsibility. 

2. Teachers were able to predict their students' responses only 
when student variability was low. Therefore, no evidence was 
provided that teachers recognized individual differences 
among students. A more accurate measurement of teacher 
sensitivity would be necessary to obtain more conclusive 
results, 

3. The majority of teachers seemed to hold opinions about 
education that were similar to those held and espoused 
by educational authorities. In addition, teachers 
generally considered those components important to 
individualized instruction that were cited in the 
literature as being important. It was evident that 
teachers were aware of the important components of 
individualization and were in general agreement with 
the premises of individualized instruction. 

Additional Information ; 

72- 109 Evaluation Design for the System-Wide Assessment of 

Individualization, 1973, George H. Olson, 85 pp. 

73- 161-1 A System-Wide Assessment of Individualized Instruction: 

Volume I - Overview and Summary, 1973, George H. Olson, 
35 pp. 

73-161-2 A System-Wide Assessment of Individualized Instruction: 

Volume II - Review of Related Literature, 1973, 
George H. Olson, 56 pp. 
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73-161-3 A System-Wide Assessment of Individualized Instruction: 

Volume III - Administrative Considerations, 1973, 
George H. Olson, 29 pp. 

73-161-A A System-Wide Assessment of Individualized Instruction 

Volume IV - Classroom Practices - Elementary and 
Secondary, 1973, George H. Olson, 106 pp. 

73-161-5 A System-Wide Assessment of Individualized Instruction: 

Volume V - Teacher Characteristics and Considerations, 
1973, George H. Olson, 30 pp, 

73-161-6 A System-Wide Assessment of Individualized Instruction: 

Volume VI - Appendices, 1973, George H. Olson, 66 pp. 
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ELEMENTARY AND SECONDARY 
SCHOOL INDICES 



Purpose : The purpose of the school index reports is to provide key 
administrators and District decision-makers with a quantitative index 
expressing the level of difficulty in administering and leading the 
educational program of each school in the District. These indices 
should aid administrators and decision-makers in evaluating each 
school community in terras of personnel and administrative problems 
and assist them in devising effective educational programs for each 
school. 



Method: The elementary and secondary indices were derived by stan- 
dardizing weighting and combining a number of measurable factors for 
each school which were related to the difficulties encountered in 
serving as an educational leader for the school. Three distinct sets 
of elementary indices were generated using the following variables: 



^1 - 


average daily attendance, 


^2 - 


average daily membership, 


^3 - 


mobility ratio. 


h - 


grade-two average California Achievement Tests 




score p 




grade-four average Comprehensive Tests of Basic 




Skills scores 


^6 - 


median parental income of the school community 



The three sets of elementary indices were then constructed by the 
following weighting formulas after all had been standardized to the 
same mean and standard deviation: 

(Index 1) y = 4X^ + 4X3 - - 

(Index 2) y = + 2X3 - X^ - X^ - Xg 

(Index 3) y = 4X2 + 4X3 - X^ - X^ 

The secondary school indices were computed by combining the 
following six variables: 

X^ - mean test scores, Iowa Tests of Educational 
Development in grades nine and twelve and 
Comprehensive Tests of Basic Skills in 
grades seven and eight. 
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X2 - attendance ratio defined as average daily 

attendance divided by average daily member- 
ship 

~ mobility ratio 

X. - number of students bussed 
4 

X^ - .enrollment 

Xg - percent minority, the ratio of the number 
of students not members of the schools' 
dominant ethnic group to total enrollment 

After standardizing each variable, a composite Index was created by 
the following weighting formula: 

Y « " X, - 2X^ + 2X^ + X, + 4X. + X^ 



Additional Information ; 

73-110 Elementary and Secondary School Indices, 1973, Clinton C. 
Schuhmacher, 18 pp- 
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Harold W. Lang, Ed.D. 
Director - 
System-Wide Testing 



TECHNICAL CHARACTERISTICS OF STANDARDIZED TESTS 
USED IN THE DALLAS INDEPENDENT SCHOOL DISTRICT 



Description of the Study ; This report presents the results of a major 
study of the technical characteristics of some of the tests employed in 
the District's System-Wide Testing Program. The source of the test data 
used in this study was the 1971-72 item data tapes for the Comprehensive 
Tests of Basic Skills (CTBS) and the California Short-Form Test of Mental 
Maturity (CTMM) . This study examined the degree of reliability of each 
test within each grade level ahd the intercorrelations of test and subtest 
scores. 



Results: 

1. The total scores on both the Comprehensive Tests of 
Basic Skills (CTBS) and the California Short-Form Test 
of Mental Maturity (CTMM) evidenced a high degree of 
reliability within all District sex/ethnic student 
subpopulations at all grade levels. 

2. Reliability increased with the age of students. 

3. Reliabilities were slightly higher for Anglo students 
than for minority students. 

4. The CTBS was more reliable for all sex/ ethnic sub- 
populations than was the CTMM. 

5. Study Skills was the only subtest of the CTBS that 
evidenced a sufficient lack of reliability so as to 
preclude that subtest's use in instructional planning 
for individual students. 

6. Correlations between total scores of the CTBS and 

CTMM were high, ranging from .68 to .87. This suggests 
that the two tests measure essentially the same global 
construct . 



Re c omm enda t i o n s : 

1. It appears that simultaneous administration of the CTBS 
and CTMM is wasteful in terms of information received 
for student time required. Thus, it is recommended that 
the CTMM be eliminated. This recommendation is made 
on the basis of the higher reliability for the CTBS, the 
high correlation between the CTBS and the CTMM, and the 
popular misconception that the CTMM is measuring 
exclusively innate ability. 
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Additional Information: 



73rl82 Technical Characteristics of Standardized Tests Used In the 

Dallas Independent Schooil District, 1912'-12y George H. Olson, 
40 pp. 

73-183 Test Administration Practices In the Dallas Independent School 
District, 1972-73, George H. Olson, 19 pp. 

73-184 Test Use and Interpretation In the Dallas Independent School 
.District, 1973, George H. Olson, 34 pp. 

72-54 Selected Characteristics of Standardized Tests Used In the 

Dallas Independent School District, 1972, George H. Olson, 23 ; 



TEST ADMINISTRATION PRACTICES IN THE DALLAS INDEPENDENT 

SCHOOL DISTRICT 



Description of the Study ; For the purpose of evaluating the extent to 
which administrative practices recommended in the System-Wide Testing 
Manual (Report No. 72-89) were followed during the fall, 19723 testing 
program, a questionnaire was sent to a stratified random sample of 
individuals involved in test administration. In addition to the 
responses to the aforementioned questionnaire, information of an 
anecdotal nature collected by the Supervisor of Group Testing and by 
Educational Testing Service is synthesized and presented • This report 
provides valuable input to District planners in organizing and imple- 
menting the future administration of the system-wide testing program. 



Results ; 

1. Approximately 15% of the survey respondents reported 
extraneous interferences during the testing sessions. 
Extraneous interferences reported included such things as 
fire drills and public address announcements. 

2. Approximately 7% of the survey respondents reported 
that some of their students did not attempt to take all 
sections of the test. 

3. Approximately 5% of the survey respondents reported 
coaching by the test administrator. 

4. Apparently some schools do not have organized procedures for 
ordering and distributing test materials as evidenced by: 

a« Numerous last-mdnute requests for materials were submitted ^ 
some of which had already been sent. 

b- Students were, in some cases, obliged to record their 

answers on answer sheets other than the one designed for 
the test they were taking. 

5. School personnel often failed to supply accurate and complete 
identifying information when returning test booklets to the 
service center. This often caused delays in test-score turn- 
around and occasionally resulted in lost scores. 

6. The report by Educational Testing Service, which involves only 
Title I schools, supports points one and two above and, in 
addition, is critical of the physical arrangements of the test- 
ing sites. This is particularly true with respect to the seat- 
ing of students and the large number of students tested in a 
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given testing session. The latter observation is validated 
by the fact that approximately 21% of the survey respondents 
repot Led administering standardized tests to large groups of 
primary students. 



Recommendations : 



1. Top management support is needed to insure tb'it standardized 
tests are administered in a professional manner. 

2. Systematic auditing of test administration procedures is 
necessary. This would include the documentation of deviant 
situations. 



3. Expanded printing and dissemination of the System-Wide 

Testing Manual to every principal, counselor, and building 
test coordinator is essential* 



4. Staff development programs in test administration should 

be designed and conducted. Given current financial limita- 
tions, these programs should be conducted by building test 
coordinators after having attended an in-depth orientation 
session presented by the Director of System-Wide Testing 
and his staff. 



Additional Information!' 



72- 54 Selected Characteristics of Standardized Tests Used in the 

Dallas Independent School District, 1972, George H. Olson, 
23 pp. 

73- 182 Technical Characteristics of Standardized Tests Used in the 

Dallas Independent School District, 1972-73, George H. Olson, 
40 pp. 

73-183 Test Administration Practices in the Dallas Independent School 
District, 1972-73, George H. Olson, 19 pp. 

73-184 Test Use and Interpretation in the Dallas Independent School 
District, 1973, George H. Olson, 34 pp. 
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TEST USE AND INTERPRETATION 
IN THE DALLAS INDEPENDENT 
SCHOOL DISTRICT 



Description of the Study : An anonymous survey questionnaire and achieve- 
ment test were answered by a 5% random sample of teachers and counselors 
drawn from all secondary schools and from 20 randomly selected elementary 
schools for the following purposes: 

1. to assess test users' level of knowledge concerning 
the interpretation of standardized test results 

2. to estimate the degree of reliability and validity that 
test users sec in test results 

3. to determine the extent and purposes for which test 
results are used in the District 

4. to .determine the extent to which test users feel test 
results are available 

5. to determine the extent and nature of modifications that 
users feel should be made in the system-wide standardized 
testing program 

This report provides valuable input to District decision-makei's in 
planning the future of the District's standardized testing program. 

Results : 

1. -District counselors evidenced a greater degree of under- 
standing of correct use of standardized tests than did 
District teachers. However, neither group performed to 
a standard that would be optimal for adequate test usage. 

Approximately 55% of the teachers and 75% of the counselors 
sampled felt that individual student scores on standardized 
tests are reliable. 

Approximately 80% of the teachers and counselors responding 
felt that the District's aptitude tests are valid for help- 
ing to identify underachievers . 

Approximately 50% of the teachers sampled and 60% of the 
counselors sampled felt that the District's standardized 
tests yield accurate measures of academic achievement and 
ability. 
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5. Over 70% of the counselors responding reported that they 
use test results to discuss student progress with parents,, 
to check classroom performance against test performance, 

to diagnose student learning deficiencies, and to aid 
students in making career choices. 

6. Over 60% of the teachers responding reported that they 
use the test results to check classroom performance 
against test performance, to diagnose student learning 
deficiencies, and to aid in planning individualized courses 
of study for students. 

7. Approximately 10% of the teachers and V4 of counselors 
responding reported never using the standardized test 
results. Approximately 50% of teachers and 75% percent 
of counselors responding reported using standardized test 
results for all students. 

8. The majority of each group of respondents agreed that the 
cost in instructional time lost for testing was worth the 
benefit gained from the results of the standardized test- 
ing program and that the standardized tests currently 
administered are appropriate for measuring achievement 

of the District's educational goals. The most frequent 
suggestions for improvement were: 

a. revise the testing schedule 

b. develop culturally unbiased tests 

c. improve testing conditions 



Recommendations ; 

1. It is obvious that a staff development program is needed 
for teachers and counselors in the use and interpretation 
of standardized test results. 

2. Some relief in the testing schedule is needed so that so 
many tests are not given at one point in time in the school 
year. 

Additional Information ; 

73-182 Technical Characteristics of Standardized Tests Used in the 

Dallas Independent School District, 1972-73, George H. Olson, 
40 pp. 
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73-183 Test Administration Practices in the Dallas Independent School 
District, 1972-73, George H. Olson, 19 pp. 

73-184 Test Use and Interpretation in the Dallas Independent School 
District, 1973, George H. Olson, 34 pp. 

72-54 Selected Characteristics of Standardized Tests Used in the 
Dallas Independent School District, 1972, George H. Olson, 
23 pp- 
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A PLAN FOR SYSTEM-WIDE TESTING 



In order for a System-Wide Testing Program to be maximally effec- 
tive, it must be continually evaluated and updated. The plan presented 
in this report is based upon systematic feedback from all persons in- 
volved in the Testing Program, a review of the literature relative to 
current and effective testing practices, the results of a recent study 
completed by the Department of Research and Evaluation on the technical 
quality of the Testing Program, and several conferences with members of 
the Principals* Advisory Committee, In addition, plans for a Criterion- 
Referenced Testing Program are integrated into the overall testing plan. 

Recommendations are as follows: 

1. Continue the system-wide administration of the 
Metropolitan Readiness Tests in the fall of grade one, 
the California Achievement Tests in the fall of grade 
two, the Comprehensive Tests of Basic Skills in the 
fall of grades three through eight, and the Iowa Tests 
of Educational Development in the fall of grade nine. 
These tests are of sufficient technical quality and 
practical utility to warrant their continued admin- 
istration. Grade equivalents and percentiles will 

be included on gummed labels . 

2. Continue to develop the District's Criterion-Referenced 
Testing Program. The emphasis for 1973-74 will be 
secondary reading. 

3. Discontinue the system-wide administration of the 
California Achievement Tests in the spring of the first 
grade. However, these tests may be administered for 
special cases and purposes. These tests are administered 
in the fall of grade two and scoring turnaround for 
spring testing precludes feedback from reaching the 
classroom until the fall semester when these tests are 
again administered to the same students. 

4. Instead of administering the Iowa Tests of Educational 
Development in the fall of the twelfth grade, administer 
them in the spring. In addition, administer the Iowa 
Tests of Educational Development in the fall of the tenth 
and eleventh grades. This should help teachers and 
counselors in diagnosis as well as enable the District 

^ better to evaluate its secondary programs. 

5. Instead of administering the California Short-Form Test 
of Mental Maturity in the fall of grades two, four, six, 
eight, and nine, administer it in the spring of grades 
two, four, six, and eight, and where needed for special 
cases and purposes. 
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This will provide the necessary data for administrative 
decision-making without overtesting students at a given 
point in time. Our reviews of the literature suggest 
that those four grades are optimal for assessing student 
growth. 

6. Administer criterion-referenced screening tests in the 
spring to all elementary students in reading (1974) and 
secondary reading (1975) , followed by diagnostic instru-- 
ments to appropriate students in the fall. This 
represents the implementation of the District's criterion- 
referenced testing program. 

7. Administer and validate a system--wide attitude-toward- 
school scale to all students in grades one through twelve 
in February. Most of the literature suggests that 
attitudes are a major concomitant of student achievement. 
This is also a high priority of the Elementary Operations 
Department. 

8. Conduct a comprehensive staff development program in the 
administration and use of standardized tests for District 
teachers and administrators. This is particularly a 
pressing need in Title I schools. 

9. Report scores on the four subtests of the California 
Short-Form Test of Mental Maturity in raw score and 
local percentiles > in addition to the traditional 
language, non-language, and total IQ. This should aid 
teachers in the diagnosis of different student- 
learning styles. 

10. Administer and score Boehm Test of Basic ConcGpts 
for kindergarten pupils in Early Childhood Program. 

11. Administer and score spring achievement posttests 
for Tif^le I schools. 
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Dallas Independent School District 
July, 1973 

1973-74 Testing Schedule for System-Wide Testing Program 



Grade 


Month 


Test 


Testing Time 


Kindergarten 


September 
10^14, 1973 


Boehm Test of Basic Con- 
cepts (BTBC) Form A, 
Booklets 1 & 2 


40 minutes 


1 


September 
10-14, 1973 


Metropolitan Readiness 
Tests - (MRT) , Form A 


1 hour 




April 23- 
May 7, 1974 


MRT - Form A, Title I 
Schools only* 


1 hour 




April 23- 
May 7, 1974 


Criterion-Referenced Test- 
Reading (CRT-R)** 


1 hour 




April 23- 
May 7, 1974 


California Achievement 
Tests - (CAT) - Title I 
Schools only. Form A, 
Level 1 


1 hour 

54 minutes 


2 


September 
17-28, 1973 


CAT - Form A, Level 1 


1 hour 

54 minutes 




April 23- 
I-iay 7, 1974 


California Short-Form 
Tes t of Ment al Ma tur i ty 
(CTMM), Level 1 


41 minutes 




April 23- 
May 7, 1974 


CRT-R 


1 hour 




April 23- 
May 7, 1974 


CAT - Title I Schools 
only. Form A, Level 1 


1 hour 

54 minutes 


3 


September 
17-28, 1973 


Comprehensive Tests of 
Basic Skills (CTBS) , Form 
Q, Level 1 


3 hours 

4 minutes 




April 23- 
May 7, 1974 


CTBS - Title I Schools 
only. Form Q, Level I 


3 hours 

4 minutes 




April 23- 
May 7, 1974 


CRT-R 


1 hour 


4 


Sep tember 
17--28, 1973 


CTBS - Level Q-2 


3 hours 
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April 23- 
May 7, 1974 


CRT-R 


45 minutes 




April 23- 
May 7, 1974 


CTBS - Title I Schools 
only. Level Q-2 


3 hours 
15 minutes 




April 23- 
May 7, 1974 


CTMM - Level 2 


43 minutes 


5 


September 
17-28, 1974 


CTBS - Level Q-2 


3 hours 
15 minutes 




April 23- 
May 7, 1974 


CRT-R 


45 minutes 




April 23- 
May 7, 1974 


CTBS - Title I Schools 
only. Level Q-2 


3 hours- 
15 minutes 


6 


September 
17-28, 1973 


CTBS - Level Q-2 


3 hours 
15 minutes 




September 
17-28, 1973 


CRT-R - Texas Assessment 


3 hours 
15 minutes 




April 23- 
May 7, 1974 


CRT-R - DISD 


45 minutes 




April 23- 
May 7, 1974 


CTBS " Title I Schools 
only. Level Q-2 


3 hours 
15 minutes 




April 23- 
May 7, 1974 


CTMl - Level 211 


43 minutes 


7 


September 
17-28, 1973 


CTBS - Level Q-3 


3 hours 
2 minutes 




April 23- 
May 7, 1974 


CTBS - Title I Schools 
only, Level Q-3 


3 hours 
2 minutes 


8 


September 
17-28, 1973 


CTBS - Level Q-3 


3 hours 
2 minutes 




April 23- 
May 7, 1974 


CTBS - Title I Schools 
only. Level Q-3 


3 hours 
2 minutes 




April 23- 
May 7, 1974 


CTMM - Level 3 


43 minutes 


0 

7 


Dep Lemper 
17-28, 1973 


XUWcl LcoLo V J. X^U UCd L.X (JLId J. 

Development - (ITED) , 
Form X-5 


*\ V^oi 1 vc 

15 minutes 



o 
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Optional 


Occupational Interest 
Inventory (Oil) 


30 minutes 


10 


September 
17-28, 1973 


ITED - Form X-5 


3 hours 
15 minutes 


11 


Se'Dtember 
17-28, 1973 


ITED - Form X-5 


3 hours 
15 minutes 




Optional 


on 


30 minutes 


12 


April 23- 
May 7, 1974 


ITED - Form X-5 


3 hours 
15 minutes 



* Tests will be administered in Title I Schools only. All tests not 

designated to be administered in Title I Schools only will be administered 
in all schools - 

** Diagnostic Criterion-Referenced Tests Reading vill be available for grades 
one through six upon need and request during the entire school year. 

*** The same form and level of tests will be administered as pretests and 
posttests . 



Additional Information : 

73-177 A Plan for System-Wide Testing, 1972-73, Harold W. Lang, 26 
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A GUIDE FOR TAB IMPLEMENTATION OF 
THE SYSIEM-WIDE TESTING PROGRAM 



The bulletins that make up this series include the following instruc- 
tions for implementing the 1973-74 Basic Comprehensive Group Testing 
Program: 

1. System-Wide Testing Schedule for 1973-74 

2. Implementation of Basic Comprehensive Group Testing 
Plan 

3. Instructions for Completion of the Class Information 
Sheet, kindergarten through three 

4. Class Information Sheet to accompany each class sec- 
tion in grades kindergarten through three only 

5. Instructions for Completing Student Test Booklets, 
grades kindergarten through three only 

6. Instructions for Coding "Student Number" Field on 
Answer Sheets, grades four through twelve 

7. Instructions for Coding "Special Code" Field on 
Answer Sheets, grades four through twelve 

8. Directions for Marking Group Header Sheet, grades 
four through twelve only 

9. Directions for Completing School Information Sheet, 
grades four through twelve only 

10. Assignment form for Building Test Coordinator 

11. Form for listing Additional Group Testing Other than 
Designated 

12. Texas Education Agency (T.E.A.) School Numbers (Same 
as location code numbers this year.) 

Five of these bulletins are being sent to each principal for dissemi- 
nation among personnel who will be using group tests. 



Additional Information: 



A Guide for the Implementation of the System- Wide Testing Program, 1973, 
Harold W. Lang, 23 pp. 
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SYSTEM-WIDE TESTING BULLETIN 



To a considerable extent, the present initiative of the Dallas Independent 
School District (DISD) is characterized by the emphasis which it places 
upon adapting the educational program to the needs of the individual 
child - individualized instruction. Since these needs are governed by the 
child's mastery of the educational content to which he has been previously 
exposed and taught, it is important to determine these content factors 
as accurately as possible. 

The Basic Comprehensive Group Testing Program of the DISD includes 
instruments which assess school readiness, scholastic aptitude, academic 
achievement, and educational development. The student's vocationally 
significant interests are appraised by option. These tests are adminis- 
tered in a systematic manner in order to provide comparable student scores. 

The present Basic Program includes recommendations from the 1970-71 
Testing Committee, the Principals' Advisory Committee, the 1973 Research 
Report No. 72-119, Selected Characteristics of Standardized Tests Used in 
DISD , -fche Research and Evaluation staff. School District personnel, and 
relevant input from a survey of practices of 25 large-city school districts • 
Results of this Testing Program are to be viewed as supplementary to 
information provided by teacher observation and judgment, school grades, 
academic motivation or known industry, and the demonstration of abilities 
in areas not normally or successfully measured by standardized tests. 

This manual has been prepared to acquaint all school personnel with the 
many facets of the System-Wide Testing Program and facilitate its 
implementation. In-depth instruction in application of test scores is 
provided for instructional and counseling personnel. 



I. Basic Comprehensive Group Testing Program 

A. General Responsibilities from Group Test Administration 

B. Additional Group Testing other than Designated 

II. Use of Large-City Norms by the DISD 
III. Job Descriptions 

A. Director — System-Wide Testing 

B. Supervisor — Criterion-Referenced Testing 

C. Building Test Coordinator 

D. Program Evaluator — Group Testing, Service Center 

E. Testing Technician — Service Center 

F. Individual Hand-Scorers 

G. Suggestive Hand-Scoring Procedures 

IV. Implementation of Basic Comprehensive Group Testing Plan 
A. Requisitioning of Testing Materials 
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B. Return of Tests for Scoring 

C. Return of Surplus Materials 

D. Testing Hints 

E. Questions You May Have Concerning Testing 

V. Supplementary Directions for Using NCS Answer Sheets 

A. Preparation 

B. Test Administration 

VI. Instructions for Coding "Special Codes" Field on Answer Sheets 

VII. Directions for Marking Group Header Sheet 

VIII. Directions for Completing School Information Sheet 

IX. Suggestions for Group Testing Administration 

A. Before the Testing Date 

B. Just before the Testing Session 

C. During the Testing Session 

D. After the Testing Session 

E. After Test Results Are Received 

F. Administration of the CTMM in Elementary School 

G. Use and Interpretation of Test Results 

H. Use of Stanines and Percentiles in Test Interpretation 

I. Supplementary Data for Use with Test Results 
J. Factors Influencing Test Results 

X. Checklist for the Administration of Standardized Tests 

XI. Checklist for Implementing the Basic Comprehensive Group Testing 
Program 

XII. Tests and Measurement Study Guide 

A. What Tests Do 

B. Understanding Test Scores 

C. Types of Tests Given 

D. Summary Statistics 

E. Suggested Data Presentation 

F. Things that Hinder Test Interpretation 

G. Factors that May Affect Student and School Performance 

H. Statistical Concepts Used in Measurement 

XIII. Standardized Tests 
XIV. Statistical Concepts 
XV. The Normal Curve 

A. Raw Scores 

B. Percentiles 
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C. Grade Ec^uivalents 

D. Standard Scores 

E. Anticipated Achievement Grade Equivalent (AAGE) 
Intellectual Status Index (ISI) 

G. Mental Age (MA) 

H. Mean, Median, and Mode 

XVI. Uses of Measurement and Evaluation 

XVII. Construction and Evaluation of Classroom Tests 

XVIII. Testing - Miscellaneous 

XIX. Criterion-Referenced Tests 



A. Developing a Criterion-Referenced Test 

B. Evaluation of a Criterion-Referenced Test 

XX. Informal Reading Inventory 

A. Basic Reading Inventory 

B. Taking an Informal Inventory 

C. Sample of Passages Selected in Determining Student's Reading 

Level 

D. Sight Vocabulary 

XXI. Suggestions for Persistent Reading Difficulties 

Appendices 

Descriptions of Tests (Appendix A) 

Group Test Reports Used in the DISD (Appendix B) 

A Glossary of Group-Testing Terms Used in the DISD (Appendix C) 

Additional Information ; 

73-177 A Plan for System-Wide Testing, 

1973-74, Harold W. Lang and 
William J. Webster, 26 pp. 

73-195 System-Wide Testing Bulletin, 1973, Harold W. Lang, 

115 pp. 
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A GENERAL DESCRIPTION OF THE DALLAS INDEPENDENT 
SCHOOL DISTRICT'S CRITERION-REFERENCED 
TEST IN READING FOR GRADES 
ONE THROUGH SIX 



Performance objectives have been selected by District administrators 
and teachers for use in the Dallas- Independent School District's criterion- 
referenced tasting program in reading for grades one through Gix. The 
objectives reflect the skills emphasized in the District's basal reading 
program. They have been specified at various levels to provide a broad 
coverage of relevant skills. Included in this report are objectives and 
word lists upon which six levels of tests have been developed for use by 
District teachers. This report has been prepared for teachers, parents, 
students, or any other person who is interested in a general description 
of the District's criterion-referenced reading program in grades one through 
six. 



Additional Information : 



72«167 Development of a Criterion--Ref erenced Testing Program, 

1972, Margot A. Olson, 44 pp. 

73-198 A General Description of the Dallas Independent School 

District's Criterion-Referenced Test in Reading for 
Grades One Through Six, 1972-73, Margot A. Olson, 49 pp. 

73--199 Evaluation of Test Items for the Dallas Independent School 

District.'^! Survey of Reading Skills, 1973, Margot A. Olson, 
80 pp. 

73-200 Technical^'Bulletin: Evaluation of Test Items ror the Sur- 

vey of R.-^^Slng Skills, 1972-73, Margot A. Olson, 215 pp. 

73-203 Evaluation of the Passages Selected for the Dallas 

Independent School District's Content - Referenced Reading 
Scale, 1973, Margot A. Olson, 42 pp. 
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EVALUATION OF TEST ITEMS FOR THE DALLAS INDEPENDENT 
SCHOOL DISTRICT'S SURVEY OF READING SKILLS 



Objectives of the Program ; The Board of Education of the Dallas 
Independent School District (DISD) has established a District-wide 
criterion-referenced testing program in reading. As an initial phase 
of the development of this program, tests have been constructed for 
use at the elementary level. The purpose of this report is to pre- 
sent technical data for the preliminary forms of tests measuring 
specific reading skills at six levels of complexity. Based upon the 
reported analyses, the tests will be revised and prepared for use 
by District teachers and evaluators. 

Sample ; Sixty classrooms from 11 different schools across the 
District were selected upon the basis of their ethnic composition 
for tryouts of the preliminary test forms. The schools represented 
were the following; 



Area 


School 


Grades 


Area I 


David Burnet 


1-6 




Sudie Williams 


1-6 


Area II 


Jefferson Davis 


1-6 




Umphrey Lee 


1-6 


Area III 


W. A. Blair 


1-6 




John Ireland 


1-6 


Area IV 


Gran Roberts 


1-6 




A. S. Johnston 


1-6 




Amelia Earhart 


1-5 




Elisha M. Pease 


1-3 




Maynard Jackson 


4-6 



Evaluation Design : Performance objectives were written to delineate 
skills relevant to the DISD's basal reading program. Item specifica- 
tions to provide greater detail as to the type of items to be produced 
were written for each objective- Word lists were compiled to provide 
appropriate vocabulary for construction of test items. Two test forms 
(Forms A and B) developed by identical procedures were constructed to 
measure performance objectives for each of the six levels. The test , 
forms included a total of 908 items to measure 99 performance objectives. 
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The tests were administered late in April, 1973. On the basis of 
location within the District, vhe examinees in half the classrooms 
received Form A, and the remainder of the examinees received Forn B. 
Classroom teachers, who administered the tests, were instructed 1:o 
follow a detailed examiner's manual provided with the tests. Pr'i.or 
to the data analysis phase of the study, opinions of the teachers 
who administered the test and of curriculum personnel were reviwed 
to eliminate test items which were faulty. Items identified by this 
means were either eliminated or set aside for revision. 

The data analysis was primarily concerned with examination of item 
statistics. In constructing the tests, it was assumed that a good 
subtest for an objective would be composed of items with similar 
difficulties and that an examinee would be likely to answer all items 
either correctly or incorrectly. For these reasons, evaluation of 
the item-means and itemi-subtest correlations were emphasized in the 
data analysis phase of the study. 

Evaluation Results : In general, it can be concluded that a large 
pool of acceptable itemis is available for compiling revised test 
formis. In considering each of the six levels, it is apparent that 
the three lower levels. Levels I, II, and III, of the tests are 
composed of subtests with less variability of item means than are 
Levels IV, V, and VI. When the increasing complexity of the con- 
tent area, as well as the increasing emphasis on comprehension at 
the higher levels is considered, this result is not surprising. 

Sufficient acceptable items are available to compose revised test 
forms at Levels I, II, and III. For nearly every objective, homo- 
geneous subtests of four to eight items can be provided using 
existing items. Due to the increased variability of the item means 
at Levels IV, V, and VI, however, greater effort will be required 
for revising the test forms. Choice distributions will be reviewed 
and a number of subtests will be lengthened to insure accuracy of 
measurement similar to that at the lower levels. 



Recommendations ; According to specific recommendations made in this 
report, tests should be revised so that they are available for use 
by teachers and evaluators during the 1973-74 academic year. To 
facilitate future test development like this, it is recommended that 
specific channels between curriculum specialists within the District 
and Department of Research and Evaluation personnel be established 
to insure relevancy of objectives and test items. 

Additional Information : 

72-167 Development of a Criterion-Referenced Testing Program, 19 72, 
Margot A. Olson, 44 pp. 
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73-198 A General Description of the Dallas Independent School 

District's Criterion^Referenced Test in Reading for Grades 
One through Six, 1972-73, Margot A. Olson, 49 pp. 

73-199 Evaluation of Test Items for the Dallas Independent School 
District's Survey of Reading Skills, 1973, Margot A. Olson 
80 pp. ' 

73-200 Technical Bulletin: Evaluation of Test Items for the 

Survey of Reading Skills, 1972-73, Margot A. Olson, 215 pp. 

73-203 Evaluation of the Passages Selected for the Dallas Indepen- 
dent School District's Content-Referenced Reading Scale, 
1973, Margot A. Olson, 42 pp. 
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PASSAGES SELECTED FOR DALLAS INDEPENDENT 
SCHOOL DISTRICT'S CONTENT - 
REFERENCED READING SCALE 



Ob^jectlves of the Program ; The Board of Education of the Dallas 
Independent School District (DISD) has established a District-wide 
criterion-referenced testing program in reading. As an initial 
phase in the development of this program, a test composed of read- 
ing selections of progressive levels of difficulties has been 
constructed for use at the elementary level. For purposes of this 
test, ability to decode and comprehend materials of a difficulty 
similar to their assigned texts has been established as the mini- 
mum criterion of acceptable performance. 



Sample ; Sixty classrooms from 11 diffe;rent schools across the Dis- 
trict were selected upon the basis of their ethnic composition for 
purposes of evaluating the reading scale. The schools represented 
were the following: 



Area 


School 


Grades 


I 


David Burnet 


1-6 




Sudie Williams 


1-6 


II 


Jefferson Davis 


1-6 




Umphrey Lee 


1-6 


III 


A. Blair 


1-6 




John Ireland 


1-6 




Oran Roberts 


1-6* 


IV 


A. S. Johnston 


1-6 




Amelia Earhart 


1-5 




Elisha M. Pease 


1-3 




Maynard Jackson 


4-6 



*Two sixth-grade classrooms were sampled. 



Evaluation Design : Reading selections from a scale developed by 
F. G. King of Florida State University were adopted for use in the 
DISD's test. Passages including at least 10 sentences and 150 words 
were selected from general reading books appropriate for all levels 
of elementary students. King modified the passages by deleting 10 
words and replacing them with four multiple-choice alternatives. 
Students were required to read the passages and select the appro- 
priate word. In addition to using nine of King's passages, an addi- 
tional passage, easier than King's least difficult passage, was 
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selected and modified by District staff. 



For purposes of equating scores from the test to students' ability 
to decode and comprehend their texts, additional tests were devel- 
oped to measure students' ability to read sampled passages from 40 
of the texts adopted by the District. Both types of tests were ad- 
ministered early in April, 1973, by the classroom teachers. The 
teachers were requested to follow detailed instructions for adminis- 
tration and to complete very short questionnaires about the tests. 

A readability formula was developed, using the structural character- 
istics and scores from the 40 sampled texts. Using the readability 
formula, a basis for assigning criterion scores for each grade level 
on the reading scale was established. In addition, the item statis- 
tics of the reading scale were evaluated. 



Evaluation Results : The data supported the assumption that the texts 
sampled for this study increased in complexity of structural charac- 
teristics and reading difficulty as they were associated with higher 
grade levels. The passages used in the reading scale likewise re- 
presented a wide range of reading complexity and difficulty. A suit- 
able readability formula, based on average sentence length, average 
word length, proportion of different words, and preparation of words 
not on the Dale List of 769 Words, was derived for use in interpreting 
scores from the reading scale. 

The item statistics and reliabilities of the reading scale passages 
revealed a number of satisfactory items and passages. Due to in- 
creased error from guessing and chance successes, the reliabilities 
of the passages tended to decrease somewhat as the passages became more 
difficult. The reliabilities of the 10 passages ranged from .56 to .88 
when calculated across all grade levels j the reliability of the entire 
scale was .96 when calculated across all grade levels. 

Reconmiendations ; Based on the criterion scores for each grade level, 
several recommendations for improving the usability of the scale were 
suggested, '.'he recommendations were made primarily to eliminate 
passages of uaarly the same difficulty and to establish a different 
criterion score for each grade level. It was also recommended that 
item statistics be closely examined as a basis for improving the 
scalability of the passages. A last recommendation was offered in 
regard to administering only part of the scale to first- and second- 
graders so that they are not frustrated by attempting very difficult 
passages . 
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Once the improvements have been completed, a satisfactory scale to 
ascertain whether students can read materials at difficulties simi- 
lar to their assigned texts should be available. 



Additional Information ; 

73-203 Evaluation of the Passages Selected for the Dallas 

Independent School District's Content-Referenced 
Reading Scale, 1973, Margot A, Olson, 42 pp. 

72- 167 Development of a Criterion-Referenced Testing Program, 

1972, Margot A, Olson, 44 pp. 

73- 199 Evaluation of Test Items for the Dallas Independent 

School District's Survey of Reading Skills, 1973, 
Margot A, Olson, 80 pp, 

73-200 Technical Bulletin: Evaluation of Test Items for 

the Survey of Reading Skills, 1972-73, Margot A. Olson, 
215 pp* 

73-198 A General Description of the Dallas Independent School 

District's Criterion-Referenced Test in Reading for 
Grades One through Six, 1972-73, Margot A. Olson, 
44 pp. 
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TECHNICAL BUT.LETIN: EVALUATION OF TEST ITEMS 
FOR THE SURVEY OF READING SKILLS 



The contents of this bulletin might more appropriately be considered 
appendices to Research Report No, 73-199^ Evaluation of the Items for the 
Survey of Reading Skills . Due to the nature and volume of materials 
necessary to document the investigation described in Report No. 73-199, 
it was considered more reasonable to include the materials in a separate 
report. Most readers will have little reason to review the Technical 
Bulletin. However, the materials will be available to readers who wish 
to study the report in depth. Of particular interest will be the copies 
of the preliminary forms of the tests. Other contents, item specifications, 
Harper Row Word List, and correspondence, vjill probably be of secondary 
interes t • 



Additional Information : 

73-200 Technical Bulletin: Evaluation of 



Test Items for the Survey of Reading 
Skills, 1972-73, Margot A. Olson, 215 pp. 



73-199 



Evaluation of Test Items for the 
Dallas Independent School District's 
Survey of Reading Skills, 1973, 
Margot A. Olson, 80 pp. 



72-167 



Development of a Criterion-Referenced 
Tc-.sting Program, 1972, Margot A. Olson, 44 pp. 



73-198 



A General Description of the Dallas 
Independent School District's Criterion- 
Referenced Test in Reading for Grades One 
through Six, 1972-73, Margot A. Olson, 49 pp. 



73-203 



Evaluation of the Passages Selected for the 
Dallas Independent School District's Content- 
Referenced Reading Scale, 1973, Margot A. Olson, 
42 pp. 
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THE DALLAS APPROACH TO SYSTEMATIC 
RESEARCH AND EVALUATION 
IN DEVELOPMENT 



Willinm J. Webster 
Deputy Assistant Superintendent 
Research and Evaluation 
Dallas Independent School District 



The basic evaluation, research, and development model 
utilized in the Dallas Independent School Di5 .rict is pre- 
sented. The approach involves the application of Stuffle- 
beam's CIPP evaluation model in conjunction with a strong 
quantitative research emphasis to provide basic information 
used in the development process. The District's longitudinal 
research and evaluation program is discussed as it relates 
to the provision of basic context data as well as to project 
evaluation support. The basic model used in project evalua- 
tion is explicated as well as the resource commitnents re- 
quired to implement the model. Explication is aided by the 
use of numerous examples drawn from the reports generated by 
the District's Department of Research and Evaluation, Finally, 
the method of communicating infomation to decision-makers is 
outlined. The process is relatively unique among public 
school systems. 



The major purposes of the Research and Evaluation Department of the Dallas 
Independent School District are to provide useful information to decision-makers 
and to serve as an accountability agent to the District's various constituents. 
The remainder of this paper discusses the strategies used to accomplish these 
purposes. 

Organizing For Evaluation 

The Departme-nt of Research and Evaluation is organized into three branches. 
The first, Systeri-Wide Testing, is responsible for the design and implementation 



Paper delivered :o a symposium at the annual r.eeting of the National Council, 
on Measurement ii Education, New Orleans, Louisiana, February 25-March 1, 1^.73. 
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of the District's system-wide norm-referenced and criterion-referenced 
testing programs. The second, System-Wide Evaluation, performs the longi- 
tudinal and cross-sectional research and evaluation necessary to supply 
major District decision-makers with information about the overall functioning 
of the District's programs. The third, Developmental Project Evaluation, 
evaluates specific developmental projects to ascertain the effects of these 
projects on developing specific student abilities in areas that generally 
need additional mediation to that which is provided by the District's general 
academic program. Figure 1 displays the organizational structure of the De- 
partment of Research and Evaluation. 

The Department of Research and Evaluation is housed in the Development 
Division. Thus, the Deputy Assistant Superintendent-Research and Evaluation 
reports to the Associate Superintendent-Development, one of the District's two 
Associate Superintendents. The Associate Superintendent-Development oversees 
all developmental activities in the District. 

Cuba and Stufflebeam (1970) suggested that a "well-tempered" evaluation 
unit should have six organizational components plus an Administrative section. 
Figure 2 shows how the Dallas unit perfoms the functions outlined by Cuba and 
Stufflebeam with the three branches enumerated above. 

Figure 2. Degree of Agreement Between The Dallas Model and The 



CIPP Model 



Function (CIPP) 



Execution (Dallas) 



Context Evaluation 



Extensive context evaluation is carried 
on by the System-Wide Evaluation Branch* 



Input Evaluation 



Input ^valuation is not done by the De^ 
partment of Research and Evaluation^ 
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Process-Product Evaluation 



Services 



Information Processing 



Reporting 



Adminis tration 



Rather it ^is done by the Planning De^ 
partTfient in conjunction with various 
project personnel. The exception to 
this is pilot testing which Guba and 
Stufflebeam include under Input Evalua^ 
tion^ but which is done under the 
umbrella of process-^product evaluation 
in Dallas. 

Extensive procesO'-product evaluation is 
done by the Developmental Project Evalua-- 
tion Branch. 

The System-Wide Testing Branch provides 
many of the services described by Guba 
and Stufflebeam. However^ each of the 
other two branches have their own per-- 
sonnel to aid the principal investigator 
on a project in devising instruments^ 
making arrangements for information 
collection^ scheduling ^ supervising in^ 
formation collection^ etc. 

The System^Wide Evaluation and Develop^ 
mental Project Evaluation Branches have 
data processing sections to aid in infor-^ 
mation processing. The statistical 
analyses^ however^ are set up by the 
principal ^' nvestigators on each project. 

All reports are written by the principal 
investigators on the various projects. 
They are the ones who are most familiar 
with the project. They interface with 
project personnel relative to results. 
The Deputy Assistant Superintendent-- 
Research and Evaluation interfaces di- 
rectly with the Associate Superintendents- 
Development^ the General Superintendent ^ 
and a oomittee of the Board of Education 
on all evaluatioyi reports. 

Administrative functions are carried on 
by the Deputy Assistant Superintendent^ 
Research and Evaluation^ his Administra- 
tive Assistant^ and the various Directors 
and Supervisors in the Department. 



The functions alluded to in Figure 2 are detailed below. 
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CONTEXT EVALUATION 

The System-Wide Evaluation Branch operates the District's Institutional 
Research Program. Webster and Schuhmacher (1971) described the program in 
detail. Basically, this program has five major functions: 

1. The provision of a baseline of status information that 
describes the domain of concern, i.e., the decision 
arena of the parent agency; 

2. The identification, creation of, awareness of, and rank 
ordering of the needs, problems, and opportunities that 
confront the parent agency; 

3. The provision of the basis for stating change objectives; 

4. The organization and highlighting of context information 
in terms of its relevance, scope, and importance; 

5. The creation of reports on context information for various 
audiences. 

Figure 3 outlines the scope of the District's Institutional Research Pro- 
gram. Also included in Figure 3 is a projection of the number of man/days 
to be spent in each endeavor during the 1972-1973 school year. These projec- 
tions were taken from "A Plan For The Use of Research and Evaluation Resources, 
1972-73" (Webster, et. al., 1972). To aid the reader in interpreting the re- 
source commitments each position type is briefly described belov;. For detailed 
job descriptions, see "Department of Research and Evaluation: Job Descriptions 
and Minimum Qualifications". 

1. Evaluator - Consultant level or above. Ph.D. or close with 
a niajor emphasis in research design ^ measurement, and 
statistics. 
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2. Programmer - At least a B.A* in computer science or 
mathematics. 

3. Research Assistant - B.A. level personnel with major 
emphasis in the Behavioral Sciences, Mathematics, 
Computer Science, or Education. 

4. Data Technician - A high school' diploma. 



Figure 3. The Scope of The District's Institutional Research 
Program 

Pro.ject Estimated Required Resources In 

Man/Days 

Ss Q) -P O 

S ^ 8 CO K 

« O CO CO 4^ O 

^ ^ ^ ^ r*^ 

S fSJ ^ c::^ 

1. Data file development and 
maintenance. An extremely 
vital activity which is pre- 
requisite to successful 
execution of most system- 
wide evaluation projects. 75 75 120 350 

2. Dropout study. Implemen- 
tation of the longitudinal 
design. Emphasis this 
year will be placed on 
identification and pre-- 
diction of dropout on -an 

individual basis. 50 35 20 65 



3. Individualization of 

Instruction (Elementary) . 
This study will be de- 
signed and implemented 
with the intention of pro^ 
viding valuable informa- 
tion to the operations 
division relative to its 
success in attaining its 

mag or priority goal* 86 25 180 74 
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4. Individualization of 

Instruction (Secondary) . 86 26 180 74 

5. Special investigations to 
satisfy ad_ hoc information 
needs of District decision- 
makers. 100 SO 50 40 

6. Measurement profiles. Ex- 
panded reporting of system-^ 
wide testing results with 
particular emphasis on pro-- 
vision of interpretive 

material. 80 75 SO 45 

7. Drug survey. Continued 
monitoring of drug usage 

among Dallas children. 35 10 8 23 

8. Measurement study. Investi-- 
gation of the utility of 
standardized tests used in 

the DISD. 22 20 20 8 

9. ITED Longitudinal study. 

An investigation of patterns 
and trends in student per- 

formance since 1962. 20 10 10 5 

10. Teacher profiles. Descrip- 
tive data relative to 
characteristics of DISD 

teachers. ^^30 SO 30 ZO 

11^ Follow-'Up study* A simple 
survey-type study intended 
to identify vocational and 
educational pursuits of 

DISD graduates. "'^"^ 20 10 10 5 

12. Teachers tuden^ character- 
istics study. A series of 
interrelated studies designed 
to identify student and 
teacher characteristics which 
interact to affect student 

achievement (Design). 150 0 55 0 
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13. Projection studies. Poipu- 
lation and enrollment pro^ 
sections for school 
attendance zones in the 
DISD. 

14. CAT-MRT Comparability. An 
investigation of the feasi- 
bility of substituting the 
CAT for the MET as a post- 
test for Title I students. 

15. Grading and reporting. A 
study of the concomitants 
and effects of current and 
proposed grading practices 
(on hold). 

16. Socioeconomic study. A 
study of the socioeconomic 
patterns of the Dallas 
community and how they re- 
late to the schools. 

17. Interaction patterns. A 
study of the interaction 
patterns among Dallas 
teachers and students. 

18. Study of Dominant Values. 
A study of the dominant 
values of various school 
communities in Dallas 
(Design) . 

19. ■ Geocoding. A study to 

investigate the feasi- 
bility of determining 
geographic coordinates 
of individual student 
residences for purposes 
of school planning. 

20. Expanded capability for 
Research Data Processing. 



25 20 20 5 

20 20 0 5 

0 0 0 0 

75 50 50 30 

75 0 120 20 

25 0 0 0 

20 20 10 71 

1 30 0 0 
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Figure A shows specific project tirae allocations. Study of these 
data should further enlighten the reader relative to the types of tasks 
performed by each position type. 



Figure 4. Specific Institutional Research Project Time Allocation^ 

Project /Task Estimated Required Resources In 

Man/ Days 



5n 
O 



4^ 

^ to 

CO CO 
CO 



I. Data File Development and 
Maintenance 



Develop Data Processing 

Specifications 
Write Computer Programs 
Coding and Hand Processing 
Computer Processing 
Write Reports 



5 
25 

40 

75 



5 
25 

40 

H 
75 



220 



220 



350 



350 



2, Dropout 

Develop Data Processing 

Spec ifica tions 
Write Computer Programs 
Coding and Hand Processing 
Computer Processing 
Write Reports 
Typey Prints and Dissemi- 

'nate Reports 



15 


7 


5 


6 


21 








5 


25 


7 




11 




5 


2 




5 


50 


35 


20 



46 



19 
65 



3. Individualization of 

Instruction (Elementavij ) 

Define Problem 
Review Literature 



5 
10 



IS 
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Develop Instruments 10 1 7 6 

Srecify Sample 3 

Si^nedule Data Collection 2 3 
Develop Data Processing 

Specifications 5 3 3 

Write Computer Programs 3 16 

Write Formal Design IS 7 
Type J Print J and Dissemi-- 

nate Design 2 3 10 

Collect Observation Data 2 106 

Collect Testing Data 1 30 

Coding and Hand Processing 1 3 44 

Computer Processing 10 0 

Write Reports 15 
'^yP^y Printy and Dissemi-- 

nate Reports _2 3 16 

86 26 180 74 

4. Individualization of 
Instruction ( Secondary) 

Define Problem 5 

Review Literature 10 IS 

Develop Instruments 10 1 7 5 

Specify Sample 3 

Schedule Data Collection 2 3 

Develop Data Processing 

Specifications 5 3 3 

Write Computer Programs 3 15 

Write Formal Design 15 7 
Typej Printj and Dissemi-- 

nate Design 2 3 10 
Collect Observation Data 2 106 
Collect Testing Data 1 30 
Coding and Hand Processing 1 3 44 
Computer Processing 10 6 
Write Reports 15 
Type^y Prints and Dissemi- 
nate Reports _2 3 15 

86 25 180 74 



5. Measurement Profiles 



Develop Data Processing 








Specifications 


20 


5 




Write Computer Programs 


10 


40 




Coding and Hand Processing 






10 


Computer Processing 


10 


20 




Write Reports 


ZO 




15 


Typej Prints and Dissemi- 








nate Reports 


10 




5 




80 


Js 


30 



40- 



5 
45 



6. Drug Survey 



Develop Instruments 3 2 6 

Specify Sample 2 

Schedule Data Collection 1 
Develop Data Processing 

Specifications 3 

Write Computer Programs 5 5 

Collect Testing Data 2 

Coding and Hand Processing 1 S 13 
Computer Processing 5 
Write Reports 25 
Typej Printj and Dissemi- 
nate Reports _3 1 i 

35 10 8 23 

7. Measurement Study 

Write Computer Programs 4 10 
Coding and Hand Processing 2 
Computer Processing 4 10 
Write Reports 13 15 
^yP^j Printj and Dissemi- 
nate Reports ^ _3 _5 

22 20 20 ' 8 



8. ITED Longitudinal Study 

Develop Data Processing 

Specifications 4 1 

Write Computer Programs 4 9 

Coding and Hand Processing 
Computer Processing 4 
Write Reports 7 
Typej Printj and Dissemi- 



9. 



nate Reports 


1 




2 


3 




20 


10 


10 


5 


Teacher Profiles 










Define Problem 


7 




3 




Develop Instruments 


2 


1 


2 


S 


Schedule Data Collection 






1 




Develop Data Processing 










Specifications 


6 


6 


4 




Write Computer Programs 




17 






Coding and Hand Processing 






2 


IS 


Computer Processing 


6 


6 


2 




Write Reports 


6 




13 




Typej Printj and Dissemi- 










nate Reports 


2 




3 


10 






lo 


30 


30 
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10* Follow-up 



Develop Instrmente 
Schedule Data Collection 
Develop Data Processing 

Specifications 
Write Computer Programs 
Coding and Hand Processing 
Computer Processing 
Write Reports 
Typej Printj and Dissemi^ 

nate Reports 



11. 

Define Problem 
Review Literature 
Develop Instrmente 
Specify Sample 
Write Formal Design 

12. Projection Studies 

Define Problem 
Review Literature 
Develop Data Processing 

Specifications 
Write Computer Programs ■ 
Coding and Hand Processing 
Computer Processing 
Write Reports 
Type^ Prints and Dissemi- 
nate Reports 



13. CAT^MRT 

Define Problem 
Specify Sample ; 
Develop Data Processing 

Specifications 
Write Computer Programs 
Coding and Hand Processing 
Computer Processing 
Write Reports 
Type^ Printj and Dissemi'^ 

nate Reports 



4 2 

2 

2 1 
5 

2 4 4 

5 

8 2 

^ _± _2 

20 10 10 5 



25 5 

40 40 

5 5 
10 

70 _S _ 

150 0 56 0 



5 
5 

2 1 
7 

3 

3 2 
9 

_ _ _2 

25 10 0 5 



1 
1 

2 4 

5 8 

4 

5 8 
5 

_l _ 

20 20 0 5 



Teacher-Student Characteristics 
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14. Socioeconomic Study 



Define Problem 5 
Review Literature 5 
Develop Instnwients 2 
Specify Sample 3 
Schedule Data Collection 5 
Develop Data Processing 

Specifications 5 5 

Write Computer Programs 40 

Write Formal Design IS 5 

Collect Observation,, Data 20 40 
Coding and Hand Processing 30 
Computer Processing 5 S 

Write Reports 15^ 



75 50 50 30 



15. In-' er action 



Review Literature 10 

Collect Observation Data 50 120 

Coding and Hand Processing 20 

Write Reports 15^ 

75 0 120 20 

16. Study of Dominant Values 

Review Literature 10 

Develop Instruments S 

Write Formal Design 10^ 



2 



i> 



17. Geocoding 



Develop Data Processing 

Specifications 5 5 

Write Computer Programs 5 10 
Write Formal Design 10 

Coding and Hand Processing ^ 10_ 40^ 

20 15 10 40 



The Institutional Research Program largely involves a vast context evalua- 
tion system. In addition to the broad i^esearch and evaluation plans previously 
outlined, the availability of a systematic, continuous, sequential, iterative, 
flexible 3 open-ended, and divisible data base provides invaluable support to 
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evaluation efforts related to the many special programs and projects of 
the District, The Institutional Research data base provides much of the 
basic information required for the evaluation of these projects and pro- 
grams and thus reduces the cost of project evaluation. 

INPUT EVALUATION 

Input evaluation is not done by the Department of P^esearch and Evalua- 
tion, Due to the nature of input evaluation, subject matter specialists are 
often required. Because of the District's policy of evaluators remaining 
independent of projects to assure objective reporting, the Department of 
Research and Evaluation hires specialists in research and evaluation method- 
ology, computer science, and psychological measurement; not subject matter 
specialists that would have to be hired specifically for and assigned to 
specific projects. This is not to say that input evaluation is not done in 
Dallas. On the contrary, it is done by the Planning Department in conjunction 
with project personnel. 

The exception to this is pilot testing. One of the services provided to 
projects by the Department of Research and Evaluation is that of pilot design 
testing. These tasks are performed by evaluators' from the Developmental Pro- 
ject Evaluation Branch. 

PROCESS-PRODUCT EVALUATION 

The Developmental Project Evaluation Branch is responsible for the bulk 
of the process-product evaluation that is done in Dallas. The major functions 
fulfilled by the Developmental Project Evaluation Branch include: 

1. The detection or prediction of defects ±j\ the procedural design 
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for a program or in its implementation; 

2. The provision of data for preprogrammed decisions; 

3. The measurement and interpretation of program attainments 
during the term of the project; 

4. The measurement and interpretation of program attainments 
at the end of the project cycle; 

5. The maintenance of a program implementation record; 

6. The determination of the concomitants of student success 
in a program; 

7. The determination of program cost-effectiveness; 

8. The organization and highlighting of project evaluation 
information in terms of its relevance, scope, and importance 
to various diverse decision-makers; and 

9. The creation of reports of project evaluation for various 
audiences* 

Figure 5 outlines the evaluation process as it relates to project evalua- 
tion in Dallas. The example for which man/days are projected is the Targeted 
Achievement In Reading Program, an evaluation of reading achievement and its 
concomitants that involves four developmental reading programs and some 11,000 



students. 



Figi4re 5. The Project Evaluation 'Process 



Man/Daye 
Budget 



TARGETED ACHIEVEMENT IN READING PROGRAM 



1. Program Objectives 



2,1 Meet with deciaion-^makera and program managera 
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to determine the program objectives. 
1.2 Refine objectives through thorough 

analysis J review of literature, ques-- 
tioning decision-^makers , etc. 

Information Regarding Program Decisions 

2.1 Using the objectives, meet with decision^ 
makers'', eta. to generate a list of the 
criticat decisions to be made concerning 
the objectives and the program. 

2. 2 Determine the types of information necessary 
to make the various decisions. 

2.3 Estimate the critical decisions and plan 

the information sources so critical decisions 
receive the most information. 

Define Measurable Objectives and Related Decisions 

3.1 Work with project personnel to mold ohjec^ 
tives so that the^j may be measured. 

5. 2 Operationalize basis for decision-making to 
relate to measured achievement of objectives. 

Flan Evaluation Dissemination 

4.1 Identify the various audiences of the evalua^ 
tion report and estimate their level of 
sophistication of the intended audience. 

Identify Measuring Instruments 

5. 1 Review objectives and decisions and evaluate 
existing instruments to determine those which 
can be employed in the evaluation. 

5.2 Determine areas where no satisfactory instni^ 
ments are available and develop complete 
specifications of instruments that are to be 
constructed. 

Instrument Development and Testing 

6.1 Develop needed instruments. 

6. 2 Test new instruments, if necessary, on a sample 
of subjects. 

'6. 2 Refine new instruments on the basis of these 
tests. 

6.4 Test administration of any non-^conventional 
instruments oy observation procedures (e.g., 
interaction analysis) . 



• t 
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7, Information Collection Scheduling 



7.1 Specify sampling procedures to be employed^ 
7c 2 Determine the schedule of observations and 

the instruments to be aSninistered at each 
observation point, 
7,3 Schedule the personnel needed to administer 
instruments* 

8m Organization of Data Analysis 

8.2 Determine various formats of data including 
card and tape format specifications at various 
stages of collection and analysis. Specify 
processing necessary to put data into correct 
format at each stage of analysis. 

8.2 Plan non- statistical analysis of data and re-- 
sources necessary to perform analysis, 

8.3 Plan statistical analysis of data and programs 
necessary to analyze data. 

8.4 Determine which programs are already written 
and are ready to use^ which programs are 
written but need modifications to handle data 
in its intended formats and which programs need 
to be written with specifications of these 
programs, 

9m Formal Evaluation Design 

9m 1 Prepare design including specification of 

9ml. 1 Objectives 

9.1m 2 Instrumentation 

9,1m 3 Analysis methodology 

9.1m4 Data collection and reporting schedules 

9.1m 5 Sampling proced.ures 

9.1.6 Data analysis schedules 

9.1.7 Final reporting schedules 

.9.2 Type J print j and collate. 
9,3 Disseminate formal designm 

10, Computer Program Development 

20.1 Develop necessary programs for analysis m 

10.2 Make necessary modification of existing 
programs . 

10.3 Run all programs to be used on sample data in 
the proper medium and format. Construct sample 
data to simulate problem in actual dxxta (mis- 
punching, missing data, etc). 
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IIo Process Evaluation 

11.1 Collect or supervise and coordinate collec-^ 

tion of process evaluation information. 30 

11.2 Prepare process evaluation information for 
analysis. 4 

12. Product Evaluation 

12.2 Collect or supervise and coordinate the collect 

tion of product evaluation information. 30 

12.2 Prepare product evaluation information for 

analysis. 20 

13. Interim Data Analysis 

13.1 Organize interim data. 10 

13.2 Perform analysis of interim data. 4 

14. Formative Evaluation Reports 

14.1 Prepare formative evaluation reports. 6 

14.2 Type J prints and collate formative evaluation 
reports. 3 

14.3 Disseminate formative evaluation reports. 1 

15. Summative Data Analysis 

15.1 Organize summative data. 5 

15.2 Perform analysis of summative data. 20 

16. Summative Evaluation Reports 

16.1 Prepare the various summative evaluation re-- 
ports for each audience including objectives^ 
findings^ and recommendations expressed in 

an appropriate manner for the intended audience. 
This preparation includes the abstract of the 
report. 15 

16.2 Have report carefully proofs-read and corrected. 3 

16.3 Type^ prints and collate the summative evalua-* 

tion reports. 5 

16.4 Disseminate the summative evaluation reports to 
project personnel and the Board Evaluation 
Committee. 1 

17. Interpretation of Reports 

17.1 Meet with project personnel to interpret reports. 2 
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4 

17.2 Meet with Board Evaluation Committee to aid 

in report interpretation.^ 1 

18. Further Report Dissemination 

18.1 Disseminate summative evaluation reports to 
District administrators and to the Board of 
Education. 2 

19. Report Feedback 

19.1 Meet with decision-^makers to obtain feedback 
regarding the report with the purpose of im- 
proving reporting activities. 3 

19.2 Prepare informal report of recommendations 
regarding reporting activities. 

Since the Department of Research and Evaluation fulfills an accountability 
function, care must be taken to assure objective evaluation. Therefore, all 
evaluation personnel, regardless of their assignment, are treated as members 
of the central research and evaluation staff. Their continued employment is 
not contingent upon continued funding of the specific developmental project or 
projects to whicV^ t:^»/ay -ai^y assigned. Experience suggests that this arrange* 
ment lea-jn. to increased objectivity on the part of evaluators. On extremely 
con^ii^oversial projects, outside auditors, generally fron Educational Testing 
Services, are used for additional credibility. Figure 6, taken from the 
District's Policy Manual, outlines the District's guidelines for project evalua- 
tion. 

Figure 6. Dallas Guidelines For Project Evaluation 

Recognizing the need to ensure the credibility of evaluation 
reports^ all project evaluation staff assigned to projects which 
the^ Department of Research and Evaluation has the responsibility 
for evaluating will be directly responsible to the Deputy Assistant 
Superintendent-Research and Evaluation. In addition to ensuring 
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the Qred'^bility of evaluation re-ports^ central oontrol 
of evaluation personnel will allow for the most efficient 
use of human vesouroes. Thus^ evaluation personnel^ re- 
gavdless of location^ will he staff members of the Research 
and Evaluation Department and as such subject to the 
following conditions: 

J. Dissemination and UHlization of Evaluation Information 

i4. Availability of evaluation information^ prior 
to its format presentation to the Board of 
Education^ will be limited to those individuals 
who have an established need to know. Normally j 
this would include project officials^ as well 
as the evaluation personnel associated with the 
project. 

B. The proprietary nature of evaluation information 
must be respected. The proper functions of 
evaluation are to provide continuous feedback to 
project officials and to furnish relevant infor- 
mation to District decision'-makers . 

C. All individuals who have access to evaluation 
information must be cognizant of its proper 
functions and of ethical considerations apply ^ 
ing to the misuse of suqh information. Evalua- 
tion personnel^ particularly^ are expected to 
process information in accord with responsible 
reporting procedures. 

II. Scope of Work 

A. What to evaluatej when to evaluate^ and how to 
evaluate will he determined by personnel from 
Research and Evaluation in consultation with 
those charged with the responsibility for making 
decisions about the operation of the project. 

B. Assignment of duties for evaluation personnel 
shall be made by the Deputy Assistant Superin- 
tendent-Research and Evaluation or his designee* 

C. All evaluation reports will be delivered simul- 
taneously to the project manager and the Deputy 
Assistant Superintendent^Re search and Evaluation. 
The Deputy Assistant Superintendent-Research ayid 
Evaluation will then forward evaluation reports 
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to the Associate Superintendent^ Development^ 
for presentation to the Evaluation Cormittee 
of the Board of Education. 

III. Employment 

A. All evaluation personnel will be reoormended 
for employment, promotion, or termination of 
employment by the Deputy Assistant Superin^ 
tendent ^Research and Evaluation after consul^ 
tation with the PrQ-CQot Director. 

B. The Deputy Assistant Superintendent-^Research 
and Evaluation will recommend placement of all 
personnel. 

C. Terms of employment will be the same as comparable 
central office positions. The employment security 
of evaluation personnel assigned to projects will 
not be dependent upon the continued implementation 
of specific projects to whi-ch they may be assigned. 

D. All leaves of absence and vacations will be subject 
to approval by the Deputy Assistant Superintendent- 
Research and Evaluation after consultation with the 
Project Director. 

IV. Local Regulations and Negotiations 

A. All evaiuators placed in locatioyis other than the 
Central Administration Building shall he subject 
to the same rules and regulations as other admin-- 
istrators in that location^ with the except'^'on of 
those items specified under Sections II an:> III 
above. 

B. All disagreements relative to the functions of 
project evaluation personnel will be negotiated 
by the Deputy Assistant Superintendent^Research 
and Evaluation and personnel responsible for 
making decisions relative to theproject.- 



Figure 7 outlines the scope of the District's Developmental Project 
Evaluation. Job types are about the same as described for System-Wide 
Evaluation. 
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Figure 7. The Scope of The District Developmental Project 
Evaluation 



Project 



Estimated Required Resources lyi 
Man/Daijs 



5 



44 

8 to 

03 CO 
CO 



Tco'geted Aahievement in 










Reading Program 


145 


67 


57 


47 


Physical Education 


35 


12 


9 


8 


Cognitive ECE 


65 


18 


21 


22 


Comprehensive Music 


46 


8 


18 


14 


Dunbar Science 


45 


9 


17 


17 


Learning Through Piano-Dallas Plan 


; 25 


7 


12 


12 


Talent Education 


40 


8 


12 


13 


Visual Art Program 


52 


12 


19 


19 


Bilingual Multicultural Education 










Program 


218 


24 


42 


26 


Summer Reading Program 


43 
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Formal evaluation designs are done for* all projects. Generally, the 
following criteria are used in judging the quality of individual evaluation 
designs. In order to be considered an adequate design, all of the following 
questions must be answered in the affirmative. 

1. Are the objectives of the program adequately stated? 

2. Are the decision situations to be served adequately defined? 

3. Are the evaluation questions of interest adequately delineated 
and do they adhere to the decision situations to be served? 

4. Are the data to be collected adequately specified and do they 
match the evaluation questions of interest? Are all questions 
adequately investigated? 

5. Are the relevant populations and sampling procedures for data 
collection adequately described? Is there reason to believe 
that the experimental and control groups are comparable? 

6. Are the instruments for data collection adequately described? 
Are they related to the objectives of the program. Are they 
valid and reliable for the population being studied? 

?• Are schedules specified for information collection? Are they 
realistic? 

8. Are formats and means for coding, organizing, storing, and 
retrieving data specified? 

9. Are data analysis procedures specified? Are the analysis 
procedures specified appropriate for providing useful infor- 
mation relative to achievement of the objectives of the program 
and the research questions of interest? 



ERIC 



187 



10. 18 the evaluation schedule present and, given staff and 

resource availability, is it realistic? 
lie Is the evaluation design likely to provide useful (i.e., 

valid, reliable, objective) information? 

12. Are there provisions made for process evaluation (i.e., for 
observing the project in operation) to determine whether or 
not it is functioning according to specifications? 

13. Are there provisions made for adequate interim evaluation? 

14. Is a schedule specified for reporting relevant information 
to specified decision-makers? Is the schedule realistic? 

15. Is the budget adequate to carry out the proposed evaluation? 

SERVICES 

Cuba and Stufflebeam (1970) assigned the following functions to the 
Services section; 

1» To develop and maintain contact with and access to a variety 
of ir.formation sources, including such disparate sources as 
the research literature's relevant substantive experts, 
examples of good practice, and the subjects within the domain 
of the master agency (e.g., teachers, childrenj^ etc.); 

2, To devise instruments for tapping each infoxrmation source; 

3, To make appropriate arrangements for the collection of infor- 
mation as needed, including original clearances, access, 
scheduling, testing, etc.; 

4, To supervise the actual information collection in the field. 
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In Dallas, the System-Wide Testing Branch performs many of the func- 
tions described above. However > both the System-Wide Evaluation Branch 
and the Developmental Project Evaluation Branch have Research Assistants 
assigned to them. The following services are performed by Research Assis- 
tants: 

1. Classroom visitation and observation. 

2. Instrument administration, 

3. Quality control on testing data. 

4. Literature review 

5. Editing and proofreading. 

6. Limited instrument design. 

7. Limited data processing. 

8. Limited evaluation report writing, 

9. Limited administrative assistance. 

Thus, although the principal investigator on a given project is responsible 
for all phases of the evaluation of that project, he has help in the afore- 
mentioned areas from the Research Assistants assigned to his Branch. All clear- 
ances are obtained at the Assistant Superintendent level. 

INFORMATION PROCESSING 

The functions of a viable information section are as follows: 

1. To process all incoming information into storable form. In 
the main this is a coding and statistical analysis function. 

2. To develop computer applications; i.e., to write relevant 
computer programs. 
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3, To store the information in various forms, 

4, To retrieve and report available inf or.T\ation in a 
variety of formats. 

Both the System-Wide Evaluation and Developmental Project Evaluation 
Branches have data processing sections to fuiiill the aforementioned tasks- 
In addition, the District's Data Processing Department provides information 
processing help when possible. Most of the Consultant level evaluators are 
also excellent programmers and do some of the more involved programming re- 
lative to the projects that they are working on. In all cases, it is one of 
the functions of the evaluator to create the specifications for data analysis, 
although a programmer may implement the analysis. 

The aforementioned arrangement is the most successful of the numerous 
that have been tried in Dallas. A central data processing staff was not 
acceptable because priorities kept getting shuffled and evaluation reports 
were never completed on time, A general Research and Evaluation Information 
Section also failed because of problems with priorities. With the present 
system, programmers are attached to specific Branches and only have to worry 
about one set of priorities, that of the Branch to which they are attached. 

REPORTING 

A specific Reports Section does not exist in Dallas, In the formative 
days of the evaluation unit such a section did exist and proved to be more of 
a detriment than an attribute. The principal investigator is the individual 
who is most familiar with the evaluation and he, therefore, is the one who 
writes the various evaluation reports and communicates the results to decision- 
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makers. Thus, the technically oriented evaluators fulfill the role of 
interfacers and avoid the situation where the entire District is wondering 
what the group of technocrats who never interact with anyone do for a 
living. 

Two basic types of evaluation reports are written (this is of course 
in addition to the numerous informal interim reports that are supplied pro- 
ject management). The first is a rather lengthy report discussing the 
evaluation in depth and detailing the methodology that was used. This report 
is for project personnel, project management, the Associate Superintendent- 
Development, the General Superintendent, and the Evaluation Committee of the 
Board of Education (functions to be discussed shortly). 

The second is a much shorter report, seldom running more than three pages. 
This report is essentially an executive summary of the longer report, Ic 
appears at the front of the longer report as an abstract, and is forwarded 
by itself to District teachers and principals, the Board of Education, and 
the news media. Figure 8 gives an example of an Executive Summary, 



Figure 8. An Executive Swmavy Of The Educator Self-Appraisal 
(ESA) Program 

Objectives of the Program : The Educator's Self -Appraisal 
Program utilizes a videotape feedback system to train and 
encourage elementajt*y classroom teachers to use more en- 
couraging verbal and non-verbal expressions^ higher level 
cognitive and affective objectives^ and open methods in 
their classroom behavior^ as well as to foster more favor-* 
able attitudes toward the teaching profession and greater 
perceived effectiveness in their teaching. The objectives 
of the program are consistent with latest findings in 
teacher education and the methods are consistent with the 
research results relative to modeling behavior. 
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Sample : Eight elementary schools were included in the 
study. The four experimental schools were Fannin^ 
Knight J Casa View^ and De Golyer^ white Crockett^ Fields 
Reinhardty and Nathan Adams made up the control schools. 

Evaluation Design : Four experimental and four control 
schools were compared relative to the objectives of the 
program. Experimental schools were schools in which 
classroom teachers participated in the ESA program. 
Control schools were schools that were similar to experi- 
mental sciiools on all relevant variables except that 
classroom teachers in those schools were not exposed to 
the program. Variables compared were teacher mastery of 
the major objectives of ESAj teacher attitudes^ student 
achievement J and student attitudes. These variables 
were assessed prior to the beginning of the ESA program 
and at its conclusion. Descriptive and inferential 
methodology were used to analyze program outcomes. 

Evaluation Results : There was evidence to support the 
conclusion that the ESA program partially met its ob- 
jectives. In the areas of Cognitive Objectives and 
Non-Verbal behav^ior^ a clear trend in favor of the 
experimental treatment waSj as was the case with teacher 
attitudes J toward planning for teaching and their own 
effectiveness. There was also evidence to suggest that 
certain teachers profited a great deal from the program. 
Teachers having relatively little teaching experience ^ 
receiving recent degrees^ and possessing a readiness to 
modify their behavior in accordance with the objectives 
of ESA profited the most from the program. 

S , 

There were two areas in which the program j as tmplemen^y^ed 
in Dallas^ appeared to need reinforcement. These arecU^ 
were those of Affective Objectives^ and Method. Ther^i 
Was scant evidence to suggest that the program even mi^ii- 
mally met its objectives in those areas. In addition^ 
there was no conclusive evidence linking the training '\ 
program to student outcomes in the areas of achievement 
and attitudes. 'U 

Recormendations of Project Evaluator to Project Manage- 
ment: TJie results of this evaluation suggest a number of 
recommendations. Firsts it seems important that increased 
emphasis be placed on teacher training in the areas of 
Affective Objectives and Method. This must be done in 
order to increase the probability of accomplishing the--'"- 
basic objectives of the training program. 
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Second^ given the lack of oonalusive evidence to suggest 
that the pvogvam did a great deal more for the maoority 
of teachers than was accomplished through mere video-- 
taping of control group teachers ^ and, given the evidence 
that suggests that the program did a great deal for 
teachers with certain characteristics , possessing the 
salient characteristics who have been diagnosed as having 
weaknesses in the areas addressed by the objectives of 
ESA^ For purposes of placement j the Research and Evaluu" 
tion Branch will provide Staff Development with pre- 
assessment norms based on the complete 1970-71 ESA sample. 

It is the evaluator's opinion that the ESA program can 
form a vital part of the District's Staff Development 
Program if implemented on a small scale with selected 
teachers. The results of this study^ plus numerous stu- 
dies reported in the literature^ suggest that teachers 
cannot be coerced into training programs of the ESA type 
with any reasonable expectation of success. In order for 
the program to be successful^ the teachers involved must 
be aware of the behaviors expected of them and believe 
that mastery of those behaviors will increase their 
effectiveness as classroom teachers, without this commit- 
ment on the part of classroom teachers involved in the 
program, the program will probably fail. There fore j in 
addition to possessing the characteristics mentioned above, 
teacher trainees should probably be volimteers. 

The cost of continued implementation of the ESA program is 
minimal. The necessary equipment is available. Thus, the 
major cost is one of teacher time. Given teachers who want 
to participate in the training program and need the skills 
that ESA stresses, the cost would appear to be xoorthwhile. 
This recommendation assumes increased emphasis on training 
in the areas of Affective Objectives and Method. 

Additional Information : 

70-^3 Design for the Evaluation of the Educator's Self- 
Appraisal Program 

70^58 The Effects of the educator *s Self-^Appraisal Program 
on Teacher and Student Behavior 



Information is forwarded to project personnel as it becomes available. 
Figure 6 listed the guidelines for providing information to project personnel* 
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In addition to the provision of information to involved decision- 
makers, all evaluation reports are taken to the Evaluation Coramilttee 
of the Board of Education, This Committee is a permanent committee 
consisting of four Board members ana allows Research and Evaluation 
personnel to interact directly with the Board on evaluation results, 
problems, and issues. Figure 9 outlines the policy statement re- 
garding the functions of the Evaluation Committee of the Board of 
Education: 



Figure 9, Funotions of the Eduoationat Evaluation Cormittee 
of the Board of Education 

The Evaluation Cormittee reviews evaluation and 
other appropriate infoimation^ monitors, the develop-- 
ment and aooountability functions of the District^ 
evaluates the total educational program evaluation^ 
research and development efforts^ and studies and 
reports the Staters legislative program. 

Funotions 

2. Continuously assess and report to the Board the 
District's needs for program research and evalua-- 
tion information; monitor^ review^ and report 
to the Board program research and evaluation 
information; and identify and report to the Board 
unmet program evaluation and research information 
needs. 

2. Identify alternatives and recommend priorities to 
the Board for providing needed evaluation and re- 
search information; assess the consequences of 
alternative approaches to providing needed evalua- 
tion and research information; and report to the 
Board recommended priorities for providing needed 
program evaluation and research information. 
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5. Report to the Board recommended priorities for 
obtaining and utilizing funds and other re- 
sources in educational program evaluation^ re- 
search and development; and receive and review 
management recommendations regarding utilizing 
of educational program evaluation j research and 
development resources. 

4. Evaluate and report to the Board the District's 
educational program evaluation^ research and 
development efforts; receive and review reports 
of educational research^ evaluation and develop- 
mentj educational audits and cost-effectiveness 
studies and reports; and report to the Board the 
Committee's evaluation of the District' s educa- 
tional development efforts. 

Probably the single most important factor in determining whether 
or not an evaluation report is useful is timeliness. An evaluation 
report that is submitted after the decision has been made is useless. 
Therefore, all evaluation reports are completed by the Department of 
Research and Evaluation prior to July 15 of a given year. Budget 
decisions are generally made by the Board in late July and early 
August. Although it is a hardship on research and evaluation per- 
sonnel, this policy has paid dividends in top management and Board 
support for systematic evaluation. 



ADMINISTRATION 

Administrative functions are outlined in Figure 10. Every supervisor on 
the staff is responsible for some administrative tasks. By the same token, 
no individual is a full-time administrator. Personnel must be practicing 
evaluators first, administrators second. In this way, it is hoped that admin- 
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Istrators In the Research and Evaluation Department will remain in close 
touch with the* methodology uid practice of evaluation. 



Figure 10. Administrative Functions 

Task Estimated Required Resources In 

Man/Daifs 





5 




nioii 




CO CO 













1. Plan and prepare the budget 
for the Department of Re- 
search and Evaluation. IS 30 10 

2. Establish^ and revise as appro-- 
priatej^ guidelines and priori-^ 
ties for the work flow of the 
Research and Evaluation Depavt- 
ment. 

2. Make input tOj review^ iind 
interact with Research end 
Evaluation members on all, 
documents produced by th^ 
Department of Research ay\d 
Evaluation. 

4. Conceptualize new uses and 
methodologies for Research 
and Evaluation activities • 

5. Represent the Department of 
Research and Evaluation at 
numerous regularly scheduled 
and ad hoc meetings with 
District decision-^makers and 
community representatives. . 

6. Assure the technical quality 
of all documents produced by 
the Department of Research 
and Evaluation. 



90 S 0 

72 0 0 

12 0 0 

75 0 0 

10 40 6 
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Manage the activities of the 
Department of Reseavoh and 

Evaluation, 75 295 26 

5. Beportj as x'equested^ on the 
status of Research and 
Evaluation activities in the 

District. 20 20 0 

9. Recruit qualified personnel 
to fill positions in the 
Department of Research and 

Evaluation. 35 10 0 

10. Make provisions for the 
professional development 
of Research and Evaluation 

staff. 116 0 0 

11. Provide technical assistance 
to evaluation sections of 

District proposals. 30 0 0 

12. Maintain communication with 
other agencies engaged in 
research and evaluation 

activities. 97 120 0 

13. Provide staff training ser^ 
vices in evaluation techno^ 
logy to operation and 
support departments of the 

District. 130 0 . 0 



REQUIRED RESOURCES 

The major resource required for the implementation qf a research and 
evaluation strategy similar to the one outlined in this paper is a General 
Superintendent that is aware of the benefits of objective data and is suppor- 
tive of such an effort. Beyond this, production is required to show other top 
administrative staff, the Board of Education (although in Dallas we are also 
blessed with some Board members who have been extremely supportive of research 
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and evaluation activities from the beginning), project managers, and other 
concerned personnel the value of objective data in making decisions. 

Beyond this, a competent research and evaluation staff is essential. 
Staff members must possess considerable expertise in. measurement, mathematical 
statistics, research and evaluation methodology, computer programming, data 
processing and analysis, and report generation. The type of analysis and re- 
porting done In Dallas requires specialists who are dedicated to improving 
education through competent research and evaluation, and who are often willing 
to give up weekends and vacations to meet the heavy requirements of the job. 

On a more technical basis, a systematic testing program including both 
standardized and criterion-referenced instruments, and sufficient computer time 
and facilities to maintain the requisite data bases and to permit the accomplish- 
ment of the considerable data anlysis required are essential. The Department 
of Research and Evaluation has access to a Burroughs System 5500 from 5:00 P.M. 
to 7:00 A.M. every night and on most weekends. Needless to say, close coopera- 
tion with the Data Processing Department is important. 

On a more abstract level, a commitment on the part of District decision- 
makers to go with objective data as the basis for decisions rather than with 
palpitations of the heart is essential. This may be the hardest resource of all 
to achieve, but if achieved, will lead to research and evaluation data actually 
being used in decisions rather than merely being viewed as interesting phenomena. 

Finally, from the standpoint of accountability, an administration that 
recognizes that all trials are not going to be successes, and that is willing 
to report both successes and failures to the public, is important. In Dallas, 
all research and evaluation reports, be they ruccesses or failures, go to the 
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Evaluation Committee of the Board of Education. This Connnittee meets in 



an open meeting, that is, a meeting which the press is free to attend. The 
results of this policy have occasionally been temporarily disasterous, but 
in the long run have proven to be beneficial to the District from a public 
accountability standpoint* 
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